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1 ELECTRONIC BOOK CONNECTION TO WORLD WATCH LIVE 

2 Related Applications 

3 This application is acontinuation-in-part of U.S. Application Serial No. 07/991,074 

4 entitled TELEVISION PROGRAM PACKAGE^G AND DELIVERY SYSTEM WITH 

5 MENUDRIVENSUBSCRIBERACCESS,filedDecember9, 1992, U.S. Application Serial 

6 No. 08/336,247 entitledELECTRONICBOOKSELECTIONANDDELIVERYSYSTEM, 

7 filed November 7, 1994, U.S. Application Serial No. 08/160,194 and PCT/US93/1 1606 

8 entitled ADVANCED SET-TOP TERMINAL FOR CABLE TELEVISION DELIVERY 

9 SYSTEMS, filed December 2, 1993, U.S. AppUcation Serial No. 08/906,469 entitled 

10 REPROGRAMMABLE TERMINALFOR SUGGESTmG PROGRAMS OFFERED ON 

1 1 A TELEVISION PROGRAM DELIVERY SYSTEM, filed August 5, 1997, U.S. AppUcation 

1 2 Serial No. 09/1 9 1 ,520 entitled DIOrTAL BROADCAST PROGRAM ORDERING, filed 

13 November 13, 1998 and U.S. Application SerialNo. 08/923,091 entitled APPARATUS FOR 

14 VIDEO ACCESS AND CONTROL OVER COMPUTER NETWORK, INCLUDING 

15 MAGE CORRECTION, filed September 4, 1997. These applications are incorporated by 

1 6 reference herein. Also incorporated by reference are co-pending U.S. Application Serial No. 

1 7 09/237,827 entitled ELECTRONIC BOOK HAVING LIBRARY CATALOG MENU AND 

18 SEARCHING FEATURES, filed January27, 1999; U.S. Application Serial No. 09/237,828 

19 entitled ELECTRONIC BOOK ELECTRONIC LINKS, filed January 27, 1999; U.S. 

20 AppUcation Serial No. 09/391,461 entitled VIDEO CONFERENCING WITH AN 

21 ELECTR0NICB00KVIEWER,filedSeptember8, 1999; and U.S. Application Serial No. 

22 09/289,956,entitledELECrRONICBOOKALTERNATIVEDELIVERYMETHODS,filed 

23 April 13, 1999. 

24 Field Of The Invention 

25 This invention relates to the distribution of audiovisual signals through communications 

26 networks such as computer networks and servers and their subsequent storage in and display 

27 on an electronic book reader. The invention has particular use with respect to global networks 
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1 such as the Internet and "World Wide Web". The invention also relates to education. 

2 Particularly, the invention provides an alternative to in-person classroom instruction. 

3 Description Of Related Art 

4 There is a constant desire to improve education and knowledge at all levels. It is 

5 thought that true human progress can only be achieved if people' s understanding of each other 

6 is improved and if people's understanding of nature and the environment is improved. 

7 Traditionally, education and knowledge have been obtained in schools from classroom 

8 instruction and from the reading of books. 

9 The disadvantage of current classroom instructional systems is that students must be 

10 physically present in the classroom to participate in the educational process. Therefore, 

1 1 students who are geographically displaced from the location of the classroom often cannot 

12 attend class instruction as often or as timely as students who are nearby to the classroom. 

1 3 The disadvantage of textbooks is that they are often not kept current with recent events 

14 or technological changes. Textbooks are usually only updated on a yearly or less frequent 

1 5 basis, while important changes may occur monthly or more frequently. Also, to save funds, 

1 6 schools may not purchase new textbooks even though the textbooks have been updated. 

17 Therefore, the new knowledge, although available, is not communicated to students, 

1 8 Recently, audiovisual presentations have begun to be used in the field of education. 

1 9 These systems may provide playback of a recording of a lecturer who provides a presentation 

20 on an educational topic. For example, students may learn about math from watching a 

2 1 videotape or television broadcast of a math professor' s lecture. Education can also occur on 

22 a more informal basis. For example, specialty channels in the United States such as the 

23 Discovery Channel® and The Learning Channel® (headquartered in Bethesda, Maryland, 

24 U.S.A.) broadcast educational programming that both entertains and educates a diverse 

25 viewership. 

26 The disadvantage of these audiovisual systems is that they are not interactive. Students 

27 are unable to ask questions, and the lecturer is unable to tailor the presentation of material to 



-2- 



Docket 5281/PTO Filu^s/Spec wpd 

1 the specific needs of the current student audience. Consequently, the needs of the students are 

2 not met. 

3 Cable and broadcast television are commonly known media which supply information 

4 to large numbers of viewers equipped with receivers known as 'television sets." By receiving 

5 a broadcast, cablecast or satellite signal, users are able to view scenes from remote locations 

6 and observe newsworthy events which occur far from the user's location. However, 

7 conventional television is a one-way media in which users cannot communicate with each other 

8 or the broadcaster. 

9 Recently, the advent of the Intemet, and the World Wide Web, in conjunction with the 

1 0 proliferation of personal computers, has allowed people to exchange information and ideas on 

1 1 aglobal and inexpensive basis. Generally speaking, the Intemet is alarge computer network 

1 2 which connects host computers. Users with a computer, modem and telephone line commonly 

1 3 call via telephone to connect with a host. The host, being in communication with other hosts 

14 (connected to other users) is able to transfer information between users. The Intemet is used, 

1 5 for example, to transfer, data files, still images, sounds and messages between virtually any two 

16 points in the world with telephone access. 

17 The use ofthe Intemet has increased dramatically since 1981, when approximately 300 

1 8 host computers were linked together. In 1989, the number of linked host computers was fewer 

19 than 90,000; but by 1993, over a million host computers were connected. Currently over 10 

20 million host computers are linked (not including the personal computers people use to access 

2 1 these hosts via modems) and as many as 40 million people around the world may have access 

22 to the Intemet medium. This number is expected to grow to 200 miUion by the end of 1999. 

23 Users on the Intemet are able to transfer text, graphics, and still pictures between 

24 remote locations. Other types of information which can be transmitted include files containing 

25 prerecorded sequences of images. To view these images, users download a large data file, and 

26 after running appropriate software, see a sequence of images on the computer screen. These 
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1 images typically are not provided in real time, and are not viewable while the user is accessing 

2 the Internet. 

3 Therefore, even though the Internet is a two-way communication medium, it is not 

4 currently being utilized to provide video information and audiovisual presentations. This is a 

5 disadvantage, in that a large number of people have been accustomed to television audiovisual 

6 presentations, and prefer an audio-video presentation to a textual or graphical presentation. 

7 Summary Of Invention 

8 The electronic book selection and delivery system is a new way to distribute books to 

9 bookstores, public libraries, schools and consumers. The technological breakthroughs of this 

1 0 invention provide a secure electronic system for both delivering selected books and receiving 

1 1 payments. The system has an unusual combination of features that provides the consumer with 

12 a daily use household appliance that has a high tech aura while being very practical, portable, 

13 and easy to use. 

14 An advantage of the system is that it eliminates the distribution of any physical object 

1 5 such as a paper book or computer memory device from any book or text distribution system. 

16 The purchase of a book becomes a PAY-PER-READ™ event avoiding the overhead, 

1 7 "middle-men," printing costs, and time delay associated with the current book distribution 

1 8 system. Published material and text such as the President' s speech, a new law, a court decision 

1 9 on abortion, or O. J. Simpson' s testimony can be made immediately available to the consumer 

20 at a nominal fee. Alternatively, books may be made available free to the end use consumer, 

21 subsidized by advertisers who sponsor books or embed advertising within the books. 

22 The system is a novel combination of new technology involving the television, cable, 

23 telephone, and computer industries. It utiUties high bandwidth data transmissions, strong 

24 security measures, sophisticated digital switching, high resolution visual displays, novel controls, 

25 and user friendly interface software. 

26 The primary components of the text delivery system are the subsystem for preparing 

27 the text for delivery and the subsystem for receiving and selecting text that was delivered. An 
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1 embodiment of the system includes additional components and optional features that enhance 

2 the system. The system may be configured for use by bookstores, public libraries, schools and 

3 consumers. In one embodiment, the system for consumer use is made up of four subsystems, 

4 namely: ( 1 ) an operations center, (2) a distribution system, (3) a home subsystem including 

5 reception, selection, viewing, transacting and transmission capabilities, and (4) a billing and 

6 collection system. Altemative configurations of the system are defined to allow for a variety 

7 of traditional and non-traditional delivery methods. 

8 The operations center performs several primary functions: manipulating text data 

9 (including receiving, formatting and storing of text data), security encoding of text, cataloging 

1 0 of books, providing a messaging center capabihty, and performing uplink functions. In one 

1 1 embodiment, the system delivers the text from the operations center to consumer homes by 

1 2 inserting text data within analog video signals. The insertion of text is generally performed with 

13 an encoder at an uplink site that is within or near the operations center. The system can use 

1 4 several lines of the Vertical Blanking Interval ( VBI), all the lines of the analog video signal, a 

1 5 digital video signal or unused portions of bandwidth to transmit text data. Using the VBI 

1 6 delivery method, the top ten or twenty book titles may be transmitted with video during normal 

1 7 programming utilizing existing cable or broadcast transmission capability without disruption to 

1 8 the subscriber' s video reception. Using the entire video signal, thousands of books may be 

19 transmitted within just one hour of air time. Nearly any analog or digital video or data 

20 distribution system may be used to deliver the text data. The text data may also be transmitted 

21 over other low and high speed signal paths including a telephone network (e.g., a public 

22 switched telephone network) having a high speed connection such as a digital subscriber line 

23 (DSL) connection and the Internet, for example. 

24 The home subsystem performs at least four functions: connecting to the distribution 

25 system, selecting text, storing text, and transacting through a communicating mechanism. The 

26 components of the home subsystem may be configured in a variety of hardware configurations. 

27 Each function may be performed by a separate component, the components may be integrated, 
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1 or the capability of existing cable set top converter boxes, computers, and televisions may be 

2 utilized. A connector, library unit and viewer unit may be used. In one embodiment, the 

3 connector portion of the home subsystem receives an analog video signal and strips or extracts 

4 the text from the video. The home library stores the text signal, provides a user friendly 

5 software interface to the system and processes the transactions at the consumer home. The 

6 viewer provides a screen for viewing text or menus and novel user friendly controls. Altemative 

7 embodiments are presented that support delivery of text using a variety of communication 

8 mechanisms. 

9 The viewing device may be a portable book shaped viewer which stores one or more 

1 0 books for viewing and provides a screen for interacting with the home library unit. A high 

1 1 resolution display is used to both read the books and to interact with the home library software. 

12 In one embodiment, an optional phone connector or return-path cable connection initiates the 

1 3 telephone calls and, with the aid of the library, transmits the necessary data to complete the 

14 ordering and billing portion of the consumer transaction. Altemative embodiments are 

1 5 presented that support ordering and billing using a variety of communication mechanisms. The 

1 6 user friendly controls include a bookmark, current book and page tum button . The billing and 

1 7 collection system performs transaction management, authorizations, collections and publisher 

18 payments automatically. 

19 A system similar to the system for consumer use may be used in bookstores, schools 

20 and public libraries. 

2 1 In one aspect of the invention, video is collected at a remote site. The term "video," 

22 as used herein, includes stereophonic or monophonic audio signals which may accompany a 

23 video signal. Additionally, "video" is used broadly herein to include still images, groups of 

24 related still images, animation, graphics, pictures, or other visual data, including textual data. 

25 The remote video information may be obtained from a video cassette, CD ROMs, television 

26 channels, one or more video cameras, or other well known sources . If video cameras are 

27 used, they may be connected to a computer so that they are remotely controllable, or they may 
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1 be oriented such that a perception of control can be created for users. The video may relate 

2 to remote sites of interest, such as a pyramid in Egypt, or the images may relate to an 

3 educational lecture being conducted at a remote site. 

4 The collected video may be transferred to a web site, either in compressed or 

5 uncompressed form. The video may be physically transported or may be transmitted through 

6 a connnunications medium to the web site. 

7 The web site may contain a storage media that stores some or all of the video. 

8 Additionally, the web site may pass camera control commands, if applicable, to the remotely 

9 controlled cameras or may simulate the remote control of a camera. One function of the web 

1 0 site is to pass video to a plurality of users, through a communication media such as the Internet, 

1 1 in response to user selections. The video passed to the plurality of users may be live video 

1 2 being fed to the web site, or may be stored video. A number of video servers may be used to 

1 3 output the video to the users through the communications media, such as the Internet. The 

14 video may be tailored by the web site for the particular user's hardware, including data 

1 5 communication equipment, or memory size, i.e. the data rate matches the highest speed which 

16 the user's equipment can handle. 

17 Users receive and display the video sent from the web site. Many simultaneous video 

1 8 pictures may be received. The quality and frame rate of the video is dependent on the user' s 

1 9 communications hardware. Users having electronic book viewers with high-speed modems 

20 or cable modems may receive higher quaUty video. The users are able to sendcommands 

2 1 and/or queries to the web site. The commands and queries are forwarded to remote locations 

22 to control remote cameras or query remotely located instructors. Altematively , the commands 

23 may cause the web site to change from among many video signals with different camera angles 

24 or locations (or to transmit a different portion of a wide angle image), causing the user to have 

25 a perception of remote camera control. The user's commands may also cause a different 

26 portion of a received wide angle image to be displayed, giving the user a perception of camera 

27 control. 
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1 In addition to video, the web site provides information, such as graphics and text, which 

2 may be related to the video or may be a separate program. This information may be 

3 automatically supplied, or provided upon user request. Therefore, the user may be provided 

4 with a comprehensive set of information concerning remote sites, remotely located documents, 

5 or other information or data, enabling the user to be quickly educated about the remote site of 

6 interest. 

7 Brief Description Of Drawings 

8 Figure lais ablockdiagramof the primary componentsof an electronic bookselection 

9 and delivery system. 

10 Figure lb is ablock diagram of an electronic book selection and delivery system that 

1 1 uses a composite video signal. 

1 2 Figure 2 is a schematic showing an overview of the electronic book selection and 

13 delivery system. 

14 Figure 3 is a schematic of a delivery plan for the electronic book selection and delivery 

15 system. 

16 Figure 4 is a block diagram of the operations center. 

17 Figure 5a is a flow diagram of processing at the operations center and uplink. 

1 8 Figure 5b is a block diagram of a hardware configuration for an uplink site. 

1 9 Figure 6a is a block diagram of a hardware configuration for a four component home 

20 subsystem. 

21 Figure 6b is a schematic of a two unit home subsystem. 

22 Figure 7 is a flow diagram of processes performed by a video connector. 

23 Figure 8 is a block diagram for an example of a library unit. 

24 Figure 9 is a flow diagram of processes performed by a library unit on the received 

25 data stream. 

26 Figure 10 is a flow diagram of processes performed by a library unit on information 

27 requests from a viewer. 
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1 Figure 1 1 is a block diagram showing the components for an example of a viewer. 

2 Figure 1 2 is a flow diagram of processes performed by a viewer on an information 

3 request from a subscriber. 

4 Figure 1 3 is a chart depicting a menu structure and sequencing of menus in a menu 

5 system. 

6 Figure 14a is a schematic of an introductory menu. 

7 Figure 14b is a schematic showing an example of a main menu. 

8 Figures 14c, 14d, 14e, 14f, 14g, 14h, 14i and 14j are schematics showing examples 

9 of submenus. 

1 0 Figure 1 5 is a schematic diagram of an electronic book system for a bookstore or 

1 1 public library. 

1 2 Figure 1 6a and Figure 1 6b are schematics of hardware modifications or upgrades to 

13 a set top converter. 

1 4 Figure 1 7 is a schematic showing a set top terminal that includes a data receiver and 

15 data transmitter. 

16 Figure 18a is a schematic of a book-on-demand system. 

17 Figure 1 8b is a schematic of an operations center supporting a book-on-demand 

18 system. 

1 9 Figure 1 9 is a diagram depicting components used for delivery of electronic books over 

20 the Internet. 

21 Figure 20 is a schematic of a page of an electronic book having electronic links; 

22 Figure 21 is a schematic of a page of an electronic book with the electronic links 

23 shown; 

24 Figure 22 is a schematic of a show links submenu; 

25 Figure 23 is an example of links in an electronic book; 

26 Figure 24 is a block diagram of an embodiment of the invention where remote video 

27 is provided to a web server by videocassette and by ordinary television. 
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1 Figure 25 is a block diagram of an embodiment of the invention where remote video 

2 is provided by remotely located cameras and a communication network carries the video to 

3 the web server. 

4 Figures 26a and 26b are a block diagrams of an embodiment of the invention using the 

5 embodiments of Figures 24 and 25 with remotely controllable cameras. 

6 Figure 27 shows remote cameras positioned around a building for perceived camera 

7 control. 

8 Figures 28a - 28B show video images from specific cameras shown in Figure 27. 

9 Figure 29 shows remote cameras deployed to follow a parade route. 

10 Figures 30a and 30b show remotely controlled cameras at a remote location. 

1 1 Figures 3 1 a and 3 1 b show a single remote camera at a remote location, where the 

12 camera has a 180 degree spherical (or other wide angle) lens. 

13 Figures 32a and 32b are block diagrams of a server platform. 

14 Figure 33 is ablock diagram of communications paths from the server site to remote 

15 users. 

16 Figure 34 shows a home page in accordance with an embodiment of the invention. 

17 Figure 35 shows a "society" page in accordance with another embodiment of the 

18 invention. 

19 Figure 36 shows a "map" page of remote camera locations throughout the world. 

20 Figure 37 shows a "watch" page containing live video feeds from five remote cameras. 

21 Figure 38 shows a page directed to determining the user's data rate. 

22 Figure 39 shows a page of an interactive lecture. 

23 Figures 40 and 4 1 show pages of an embodiment of the invention that combines live 

24 video, prestored video, graphics, and interactive questions. 

25 Figure 42 shows a flow diagram of a method of automatically monitoring and panning 

26 an area using perceived camera control. 
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1 Figure 43 is an exemplary screen display of the present invention, showing video and 

2 also showing video data. 

3 Detailed Description Of Invention 

4 Figure la shows an electronic book distribution system 100 that may be used for 

5 distributing an electronic book. A content provider 110 may publish hard copy versions of 

6 books or other printed media including newspapers, magazines, and product catalogs, for 

7 example. The content provider 1 lOmayconvertprintedmaterialstoanelectronicformatand 

8 provide the electronic formatted materials to a distribution point, or center 1 20, over uplink 

9 path 1 15. The uplink path 1 15 may be a wired or awireless path. Theuplinkpath 1 15 may 

10 be ateleconmiunications network, for example. Theuplinkpath 115 maybeasatelliterelay 

1 1 path or a wireless telephone path. Theuplinkpath 1 15 may involve providing electronic books 

12 to the disti-ibution center on a fixed media, such as a CD-ROM, for example. 

1 3 In Figure 1 a, the content provider 1 10 and the distribution center 1 20 are shown as 

14 separate components of the electronic book distribution system 100. However, the content 

1 5 provider 1 10 and the distribution center 1 20 may be co-located. 

1 6 The distribution center 1 20 may convert printed matter into an electronic format. 

17 Alternately, the distribution center 120 may receive electronic files fix)m an outside source, such 

18 as tiie content provider 110. The distribution center 120 may process and store electronic 

19 books. 

20 The distribution center 120 distiibutes electronic books. The distribution maybe, for 

2 1 example, over distribution path 1 25, distribution network 1 30, and distribution path 1 35 to an 

22 electronic book subsystem or terminal 140, which may include an electronic book viewer (not 

23 shown). The terminal may also be a television, a set top terminal, apersonal computer, or 

24 similar device. An apparatus and method for distributing electronic books is disclosed in 

25 greater detail later. The distribution network 130 maybe an electronic book store, an Intemet 

26 web site, a wired or wireless telecommunications network, an intranet, a radio program delivery 

27 system, a television program delivery system, including cable television, satellite television 
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1 broadcast, and over-the-air broadcast, for example. The electronic book distribution network 

2 130couldincludedirectdeliverythroughamaildeUveiysystemof electronic boo 

3 media, such as a CD-ROM, for example. 

4 Figure lb shows components of an electronic book distribution system 1 70 using a 

5 television program delivery system to distribute electronic books. 

6 In the embodiment shown in Figure lb, the components of the electronic book 

7 selection and delivery system 170 are an encoder 174, a video distribution system 178, a 

8 connector 1 82, and a text selector 1 86. The encoder 174 places textual data on a video signal 

9 to form a composite video signal. Although the composite signal may contain only textual data, 

1 0 it usually carries both video and textual data. A variety of equipment and methods may be used 

1 1 to encode text data onto a video signal. The video distribution system 178 distributes the 

1 2 composite video signal from the single point of the encoder 1 74 to multiple locations, which 

1 3 have connectors 1 82. The connector 1 82 receives the digital or analog video signal from the 

14 video distribution system 178 andseparates, strips or extracts the text data from the composite 

15 videosignal. Ifnecessary, the extracted text data is converted into a digital bit stream. Thetext 

16 selector 186 works in connection with the connector 182 to select text. 

17 Using the connector 182 and text selector 186 combination, various methods of 

1 8 selecting and retrieving desired text from a composite or video signal are possible. Text may 

1 9 be preselected, selected as received or selected after being received and stored. One method 

20 is for the connector 1 82 to strip or extract all the text from the video signal and have the text 

2 1 selector 1 86 screen all the text as received from the connector 1 82. The text selector 1 86 only 

22 stores text in long term or permanent memory if the text passes a screening process described 

23 below. 

24 Figure 2 shows another embodiment of an electronic book selection and delivery 

25 system 200. The delivery system 200 includes: an operations center 250 including an upHnk 

26 site 254, a video distribution system 208, an electronic book device, or home system 258 

27 including a video connector 2 1 2, a library 262, a viewer 266, and a phone connector 270, 
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1 telephone system 274, an Internet web site 279 and a billing and collection system 278. Also 

2 as shown in Figure 2, the home system 258 may include connections to a television 259 and 

3 a personal computer 26 1 may be used to display menu screens, electronic books, electronic 

4 files, or any other information associated with the electronic book deh very system 200. In 

5 addition, the television 259 and the personal computer 26 1 may provide control functions that 

6 replicate and supplement those of the viewer 266. 

7 The operations center 250 receives textual material from outside sources 282 such as 

8 publishers, newspapers, and on-line services. Altemately, the outside sources may maintain 

9 electronic books at the Internet web site 279. The outside sources 282 may convert textual 

10 and graphical material to digital format, or may contract with another vendor to provide this 

1 1 service. The operations center 250 may receive the textual and graphical material in various 

1 2 digital formats and may convert the textual material to a standard compressed format for 

1 3 storage. In so doing, the operations center 250 may create a pool of textual material that is 

14 available to be delivered to the home system 258. The textual material maybe grouped by 

15 books or titles for easy access. 

16 As used herein, "book" means textual, graphical or video information. Such 

1 7 information may be contained in any novels, encyclopedias, articles, magazines, newspapers, 

1 8 catalogues, periodicals, or manuals or any portion or section of the above. The information 

1 9 may also be provided as a video clip, for example. The term "title" may represent the actual 

20 title assigned to a book, a video program, or any other designation indicating a particular group, 

2 1 portion, or category of textual information. The title may refer to a series of related textual, 

22 video or graphical information, a grouping of textual, video or graphical information, or a 

23 portion of textual, video or graphical information. For example, "Latest Harlequin Romance", 

24 'TouraiildReadingBooks(Agesl042);'"Encyclopedia*BRITANMCA 

25 Speech," "Instruction Manual," "Schedule of 4th of July Events " "Pet Handbooks," "Roe v. 

26 Wade," and "The Joy of Cooking," are suitable titles. Also, the title may be a graphical symbol 

27 or icon. Thus, a picture of a wrench may be a title for a repair book, a picture of a computer 
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1 a title for a computer book, a graphical symbol of a telephone a title for a telephone book, a 

2 drawing of a dagger a title for a mystery book, a picture of a bat and ball a title for a sports 

3 book, and a picture of tickertape a title for a business book. 

4 The operations center 250 includes the uplink site 254 for placing the text or other 

5 information onto a telecommunications signal and sending the telecommunications signal into 

6 a distribution system. The upHnk site 254 may include an encoder 204 (not shown in Figure 

7 2) to encode the text onto the teleconununications signal. 

8 Many analog and digital video distribution systems may be used with the electronic 

9 book deUvery system 200, such as cable television distribution systems, broadcast television 

10 distribution systems, video distributed over telephone systems, direct satellite broadcast 

1 1 distribution systems, and other wire and wireless video distribution systems. Nearly any 

1 2 distribution system which can deliver a telecommunications signal, including a video signal, will 

13 work with the electronic book delivery system 200. It is also possible to distribute the 

1 4 electronic book without using a telecommunications signal as described in the embodiments 

15 presented in Section Vn below. 

1 6 The home system 258 performs five functions: (1) connecting with a video distribution 

17 system; (2) selecting data; (3) storing data; (4) displaying data; and (5) handling transactions. 

1 8 An important optional function of the home system 258 is communicating using, in one 

1 9 embodiment, a telephone communication system 274. The home system 25 8 may be made up 

20 of four parts: a video connector 2 12 or similar type of connector for connecting with the 

2 1 distribution system 208, a library 262 for storing and processing, a viewer 266 for viewing 

22 menus and text and a telephone connector 270 for connecting with a telephone communications 

23 system 274. Additional embodiments are presented in Section VE that address alternative 

24 conamunication mechanisms. 

25 The billing and collection system 278 may be co-located with the operations center 250 

26 or located remote from the operations center 250. The billing and collection system 278 may 

27 be in communication with the home system 258 using telephone-type communication systems 
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1 (for example 274). Any of a number of communication systems as presented in Section YE, 

2 such as a cellular system or the Internet, will operate with the billing and collection system 27 8 . 

3 The billing and collection system 278 records the electronic books or video, or portions of text 

4 or video that are selected or ordered by the subscriber. The collection system will charge a 

5 subscriber's credit account or bill the subscriber. In addition, the billing and collection system 

6 278 may monitor that amount due to publishers or other outside sources 282 who have 

7 provided textual data or other services such as air time to enable the text delivery system 200 

8 to operate. 

9 Also shown in Figure 2 is an intranet 279 ' . The intranet 279 ' may be used as a part 

1 0 of a private distribution network for distributing and circulating electronic books. For example, 

1 1 aumversitylibrarymayusetheintranet279' to circulate electronic books to university students 

12 and professors. 

13 Figure 3 is an expanded overview of a delivery plan 301 for the electronic book 

14 delivery system 200. It is a comprehensive delivery plan 301 to support various types of users 

1 5 and various billing systems. Figure 3 shows that pubUshers 282 may provide text transfer 302 

1 6 to the operations center 250' and receive payments 306 from the billing and collection system 

17 278'. A separate channel upUnk site 254' is shown in this configuration receiving data 3 10 

1 8 from the operations center 250' . The operations center 250' has three separate sections (318, 

1 9 322, 326) one for text receiving, formatting and re-entry 3 1 8, a second for security encoding 

20 322 and a third section for catalog and messaging center functions 326. 

21 The collection and billing system 278' shownhastwo sections (330, 334)onefor 

22 transaction management, authorizations and publisher payments 330, and the other for 

23 customer service 334. The customer service section 334 provides for data entry and access 

24 to customer account information. Transaction accounting information 338 is supplied to credit 

25 card companies 342 by the transaction management section 330 of the billing and collection 

26 system 278'. The credit card companies 342 provide billing 346 to customers either 

27 electronically or by mail. 
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1 Methods for communicating between the subscriber base 348 and the billing and 

2 collection system 278 ' include: by telephone switching 350 alone, cellular switching 354 and 

3 telephone switching 350 combined, and by use of the cable system 358 and the telephone 

4 switching 350. The system shown supports both one-way 362 and two-way cable 

5 communication 366 with subscribers. Additional communication methods are presented in 

6 Section VII. Public libraries and schools 370 as well as bookstores 374 may use the delivery 

7 system 301. 

8 PubUc libraries and schools 370 could have a modified system to allow the viewer to 

9 be checked-out or borrowed while bookstores 374 would rent or sell the viewer and sell 

1 0 electronic book data. The bookstores 374 as well as the pubhc libraries and schools 370 may 

1 1 be serviced by cable 378. Optional direct broadcast systems (DBS) 382 can also be used 

12 with the system 200. 

13 L The Operations Center 

14 Figure 4 is a schematic of the operations center 250, which includes the uplink 254, 

1 5 The operations center 250 may gather text or books by receiving, formatting, storing, and 

1 6 encoding. A data stream 302 containing text may be received at the operations center 250 by 

17 a data receiver 402. The data receiver 402 is under the control of a processor 404. After 

1 8 reception, the data stream is formatted using digital logic for formatting 406 which is also under 

19 the control of the processor 404. If any additional text is generated at the operations center 

20 250 locally for insertion into the distributed signal, the text generation is handled through text 

21 generator hardware 410, which may include a data receiver and a keyboard (not shown). 

22 Following processing by the text generator 4 1 0, the additional text can be added to the text 

23 received by the combining hardware 414 that includes digital logic circuitry (not shown) 

24 The processing at the operations center 250 is controlled by a processor 404, which 

25 uses an instruction memory 416. The processor 404 and instruction memory 416 may be 

26 supplied by a personal computer or mini-computer, for example. To perform the catalog and 
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1 messaging functions, the operations center 250 uses acatalog and message memory 420 and 

2 the text generator 410 if necessary. 

3 The data stream of text, catalog and messages may be encoded by security module 

4 encoding 424 prior to being sent to the uplink module 254. Various encoding techniques may 

5 be used by the security encoding module 424 such as the commercial derivative of NS A's 

6 encryption algorithm (Data Encryption System (DBS)) and General Instrument's DigiCipher H. 

7 Following encoding, the encoded text may be stored in text memory 428 prior to being sent 

8 to the uplink 254. A &st-in-first-out text memory arrangement may be used under the control 

9 of the processor 404. Various types of memory may be used for the text memory 428 

1 0 including RAM. The operations center 250 may use file server technology for the text memory 

1 1 428 to catalog and spool books for transmission as is described below. The operations center 

12 250 may also store the electronic book as compressed data files. 

13 In an embodiment, to transmit textual data, the distribution system 208 (see Figure 2) 

14 may use high bandwidth transmission techniques such as those defined by the North American 

15 Broadcast Teletext Standard (NABTS) and the World System Teletext (WST) standard. 

1 6 Using the WST format (where each line of the Vertical Blanking Interval contains 266 data 

17 bits), a four hundred page book, for example, maybe transmitted during regular television 

1 8 programming using four lines of the Vertical Blanking Interval at a rate of approximately one 

1 9 book every 1 . 6 minutes (63,840 bits per second). Alternatively, books may be transmitted 

20 over a dedicated channel, which interrupts programming so that 246 lines of video can be used 

2 1 to transmit approximately 2,250 books every hour (3.9 Mbits per second) . A teletext type 

22 format is the simplest but possibly the slowest text format to use with the electronic book 

23 delivery system 200. In either event, an encoder 204 may be used at an uplink site 254 to 

24 insert textual data into the analog video signal. In many other respects, the delivery of the 

25 textual information may be completed using an existing cable television plant and equipment. 

26 Alternative transmit formats and delivery systems are presented in Section VII. 
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1 Figure 5a is a flowchart of steps involved in processing text from the publisher or 

2 provider 282 that may occur at the operations center 250. As shown in block 500, the 

3 publisher 282 processes data files of text for books, compresses, encrypts and sends the data 

4 files to the operations center 250 or uplink 254. Text files for books may be sent one book 

5 at a time. As shown in block 504, the uplink 254 or operations center 250 receives and 

6 processes the data stream from the publisher 282. Generally, part of this processing includes 

7 encryption and error correction. Text files may be deUvered for receipt by multiple home 

8 subsystems simultaneously, or to a specific individual home subsystem. 

9 In Figure 5a, the electronic books are distributed to consumers using a video 

10 distribution system such as acable television system. However, the electronic books may also 

1 1 be packaged as data packets and distributed over other telecommunications networks such as 

12 a digital wireless telephone network, for example. 

13 In one embodiment, as shown in block 508, files are broken into smaller packets of 

14 information. Header information is added to the packets. The bit stream is converted from a 

1 5 serial digital bit stream to an analog bit stream that is compatible with an NTSC video signal. 

1 6 Block 5 1 2 shows the switching of analog data into the video lines of a video signal. The analog 

17 data may be placed either in the VBI or the active video lines. In some instances, unused 

18 portions of bandwidth (such as 5-40 MHZ, 70-75 MHZ, 100-109 MHZ or other guard 

1 9 bands) may be used instead of the video lines. Alternate transmission methods are presented 

20 in Section VH. 

2 1 Figure 5b is an example of a hardware configuration to perform some of the frmctions 

22 for blocks 508 and 5 1 2. A video feed 5 1 6 is received and processed through a sync stripper 

23 520. The stripped sync signal 532 is used by the digital logic control 524. The digital logic 

24 control 524 receives the sync signal 532 and a serial digital bit stream 528 for processing. The 

25 digital logic control 524 passes the serial digital bit stream to the Digital to Analog converter 

26 536 and outputs a control signal 540 for the video switch 544. The video switch 544 integrates 
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1 the video feed 516 and analog data stream 548 into a video feed with analog data signal 

2 inserted 552. 

3 As an alternative to cable, broadcast or other television delivery methods, the public 

4 telephone system may be used to transmit books to the subscribers. An average book would 

5 take about 7 minutes to transmit over the public telephone system. Using the telephone system, 

6 it is not necessary to combine video and text into a composite signal. In most other respects, 

7 the operations center would remain similar whether text delivery was by telephone or cable. 

8 File server technology (such as that described in U.S. Patent No. 5,262,875, entitled 

9 audioAtoeofo^ SER\nERiNCLUDiNGDECOMPRESSiON/PLAYBACK issued to Mincer, 

10 et al, and, U.S. Patent No. 5,218,695, entitled RLE SERVER SYSTEM HAVING fflGH-SPEED 

1 1 WRITEEXECUTION, issued to Noveck, etal., incorporated herein by reference) maybe used 

12 at the operation center with a telephone system text delivery method. 

1 3 As another alternative to cable, television, and telephone system dehvery, the public 

1 4 telephone system may be used to provide access to the Internet, where the Internet web site 

1 5 279 may be accessed. Electronic books may be ordered, paid for, and delivered directly from 

16 the Internet web site 279 over the telephone system. 

1 7 When a wireless telephone network is used to distribute electronic books, or otherwise 

1 8 communicate with the home system 258, the home system may receive data using any one or 

1 9 more standard protocols including time division multiple access (TDMA), code division multiple 

20 access (CDMA), Global Systems for Mobile Communications (GSM) and Advanced Mobile 

2 1 Telephone System (AMPS) protocols. 

22 In any delivery system using the telephone system, individual subscribers may increase 

23 the electronic book deliver rate by incorporating high speed modems or other communications 

24 devices such as an Integrated Services Digital Network (ISDN) connector, or by use of a 

25 Digital Subscriber Line (DSL). 
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1 n. The Home Subsystem 

2 The hardware configuration for a four component home system 258 is shown in Figure 

3 6a. Figure 6b shows a hardware configuration for a two component home subsystem. The 

4 home system 258 performs several functions, such as receiving data and video transmissions, 

5 stripping (or extracting) the data from the video signal, screening and storing the data, providing 

6 user friendly interface controls and software, displaying menus and text, processing 

7 transactions, initiating telephone calls and transmitting billing data. Various hardware 

8 configurations may be utilized to achieve the desired functions of the home system 258. For 

9 example, as shown in figure 6b, the home system 258 can be configured to utilize the leception 

10 and channel tuning capability of the current installed subscriber base of cable converter boxes 

1 1 and televisions 60 1 and networked computers . The home system 258 can also be designed 

12 as an advanced set top terminal converter box with menu generation capability, electronic 

1 3 memory and a telephone modem as described in section V below. Alternatively, the home 

1 4 system 258 can be configured to support altemate delivery and ordering methods as described 

15 in Section Vn. 

1 6 The electronic components, which make up the home system 258 can be arranged in 

17 a variety of way s . In the four unit subsystem of figure 6a the viewer 266 and library 262 are 

1 8 wired together while the remaining components communicate through RF transceivers 604. 

19 In a simple version of the home system 258 there are only two units, a library 262 and a viewer 

20 266. Figure 6b shows a two unit home system 258 with certain optional features. 

2 1 The viewer 266 is generally equipped with a high resolution viewing area 602, digital 

22 logic (including a key 605, security 606, and a microprocessor 621), video graphics control 

23 and memory 607, power supply circuitry 602 (not shown), an optional battery 603 and an 

24 optional RF transceiver 604. In a two unit arrangement, the library 262 contains the connector 

25 function to the electronic book distribution system 208, connector function to a public 

26 telephone communications system, and memory 600 (which may be removable and portable 

27 600'). More specifically, the library 262 would include data stripping functions 617, digital 
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1 logic 609, memory storage 600, power circuitry 610, optional connections 6 1 1 (including 

2 cellular or PCN 611'), optional battery (not shown), optional tuner module 613 and an optional 

3 RF transceiver 604. The connector 2 12 and the public telephone system connection 270, as 

4 well as the removable portable memory unit 600 of the library 262 may be broken out into 

5 separate components. (Figure 6b shows a removable portable hard disk memory 600' with 

6 removable cartridges 6 14,) Finally, the home system 258 may include an attached keyboard 

7 267 or a wireless keyboard 268. Both the attached keyboard 267 and the wireless keyboard 

8 268 may be used to communicate with the viewer 266 (not shown) or the library unit 262. The 

9 wireless keyboard 268 may communicate using radio frequency (RF) signaling, for example. 

10 In an altemate arrangement, all functions of the home system 258 may be incorporated 

1 1 into a single unit. The functions of the library 262, for example, may be carried out by a card 

12 or chipset in the viewer 266. All the communications devices needed to couple the home 

1 3 system 258 to various telecommunications networks may also be incorporated into the viewer. 

14 All interfaces between the home system 258 and the subscriber may be included with the 

15 viewer 266. In this embodiment, the viewer 266 may include a communication device for 

1 6 receiving inputs from a separate keyboard. The viewer 266 may also include abuilt-in video 

17 camera 608" that maybe used to transmit images of the subscriber. Using the transceiver 608, 

18 the camera 608 " and the speaker/microphone 608 ' , the subscriber may use the viewer 266 

19 for video conferencing, for example. 

20 Therefore, the home system 258 may have as many as five separate components, 

2 1 which communicate with each other. The two, three, four or five separate components which 

22 make up the home subsystem can communicate with each other in a variety of ways, including 

23 hardwired connection 615, RF transceiver 604 and other wireless methods. 

24 RF communications may be used in the home, allowing separate components to be 

25 located throughout the home without restriction. The data communicated between the units 

26 may be secure data. In addition, the library 262 may provide power to the viewer 266 through 

27 the hard wire conmiunication hnk 615. 
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1 To receive and strip data from a video signal at the consumer' s home, a device such 

2 as a cable interface device or cable connector 212 is used. The cable connector device 

3 includes a tuner 613, while the cable interface device makes use of existing tuning equipment 

4 in the home. In either configuration, data is stripped from the video signal and stored at the 

5 subscribers location in the library 262. The phone connector 270, optional connector 611, 

6 and modular connector 701 initiate communications and transmit ordering and billing 

7 information to the operations center 250 or billing and collection system 278. A digital 

8 connector 6 1 9 is provided to communicate digital information with the set top 60 1 . The library 

9 262 is the inteUigent component of the home subsystem, incorporating the hardware and 

1 0 software necessary to store the text data, generate menus and effect the purchase transactions. 

1 1 In addition to an RF transceiver 604, the home library 262 also includes the necessary j acks 

1 2 and connections to allow the system to be connected to the viewer 266. As shown in Figure 

1 3 6b, the library 262 communicates the text data to the viewer 266 in a secure format, which 

14 requires a key 605 for decryption. The text may be decrypted page by page just before 

15 viewing. 

16 a. The Video Connector 

1 7 Figure 7 shows the flow of the processes performed by the video connector 212. The 

1 8 video connector 212 receives the video signal 608, tunes to the channel containing the text data 

19 612, strips the text data from the video signal 6 1 6, and conrniunicates the text data stream to 

20 logic components in the Ubrary 620. 

2 1 The connection to the video distribution system may be a cable connector to a cable 

22 television delivery system, as shown in Figure 6b. The cable connector includes a data stripper 

23 circuit 617, which accepts video input from either a set top converter, TV or VCR 60 1 , or an 

24 optional tuner block 613 that receives the CATV signal through the cable connector 212'. The 

25 data stripper circuit 617 strips data out of the video, and outputs a digital bit stream to the 

26 digital logic portion 609 of the library unit 262. The data is embedded in the video signal either 

27 in the vertical blanking interval or the active video portion in an encrypted and compressed 
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1 format. The data stripper circuit 6 1 7 can be placed inside the set top converter box 60 1 , TV, 

2 or in the library unit. The data stripper circuit 617 outputs the digital bit stream to be used by 

3 the library digital logic 609. 

4 The video connector 2 1 2 may also contain a channel tuner module 6 1 3 that can tune 

5 to the video channel and provide access to the video that contains the data to be stripped. 

6 Using the optional tuner module 6 1 3, a set top converter, VCR, or TV tuner is not needed in 

7 the home subsystem. The optional tuner module 613 would instead receive the CATV signal 

8 directly through the cable connector 2 1 2. Additional connector options, which allow for the 

9 receipt of text files using alternative deUvery methods, are presented in Section Vn. This 

10 ubiquitous access is provided using the modular connector 700 as depicted in Figure 6b 

11 b. Library 

12 An embodiment of the library 262 for a two unit home subsystem is shown in both 

1 3 Figure 6b and Figure 8 . The embodiment shown includes the following optional parts : the 

14 video connector 212, phone connector 270, RF transceiver 604, and battery pack 624 in 

1 5 addition to a removal portable memory 600' , microprocessor 628, instruction memory unit 

16 632, digital logic 636, and power unit 640. 

1 7 The library 262 contains a digital logic section 609 (not shown in Figure 8) which 

1 8 includes the microprocessor 628, the digital logic 636 and the instruction memory unit 632. 

1 9 The microprocessor 628 may be a secure microprocessor such as the Mot SC2 1 device sold 

20 by Motorola. The digital logic section 609 will receive the serial digital bit stream from the data 

2 1 stripper circuit 6 17 and process the data. Error correction will also be performed by the digital 

22 logic section 609 and the data will be checked for proper address. If the address of the data 

23 is correct and the library 262 is authorized to receive the data, the data will be transferred to 

24 die memory storage unit 600, 600'. Authorization to receive the data is provided by the cable 

25 headend or another distribution point. An authorization code may be sent in the serial digital 

26 bit stream. The digital logic section 609 will send appropriate text and graphical data to the 
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1 memory storage unit 600, 600 ' . It transfers this data in a compressed and encrypted format 

2 and the data remains stored in a compressed and encrypted format. 

3 i. Memory Storage Unit 

4 The memory storage unit of the library may be a removable portable memory unit 600' 

5 (as shown in Figures 6a, 6b and 8). A variety of options are available for memory storage: a 

6 hard disk drive, such as an 80 megabyte, a 200 megabyte, a hard disk with removable platters, 

7 and CD ROM. Referring to Figure 6b, a hard disk drive unit 600' , which contains removable 

8 platters, may also be used. This would provide virtually unlimited library storage capacity. 

9 Data will be stored in the memory storage unit in a compressed and encrypted format. As is 

1 0 also shown in Figure 6b, the data may also contain a key or unique ID number that matches 

11 the ID or key of the viewer 266. This matching of a unique key or ID number prevents 

1 2 unauthorized transfer of text data from the memory storage unit to an unauthorized viewer. 

13 Small memory devices such as smart cards, electronic memory cards or PCM CIA cards 

14 (personal computer memory card industry association) may also be used to store the data. 

15 ii . Power Circuitry 

16 As shown in figures 6b and 8 , ttie library 262 will accept power from AC wall power 

17 610, DC power 640, or optional battery power 624. The power circuitry 610, 640 may 

1 8 provide all the voltage necessary from either the battery 624 or AC unit for the various circuitry 

19 in the library. The power circuitry 6 1 0, 640 may also provide power to the viewer through a 

20 singledatacable when connected to the viewer. The power circuitry 610, 640 will recharge 

2 1 the battery using AC power when in operation. With the optional battery unit 624 installed, the 

22 library 262 becomes a portable unit and can still provide power to the viewer 266. In order 

23 to extend battery life, power conservation measures may be utilized, such as shutting down the 

24 memory system when not in use. When the viewer unit 266 is being utilized and the library 

25 circuitry is not being utilized, virtually all power may be shut down to the library 262. 
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1 iii. Connection to the Public Telephone System 

2 In an embodiment, the connection to the telephone system may be provided by a 

3 connector device 611, which consists of a modem. Various available modems may be used 

4 to perform this function. As shown in Figure 6b, cellular phone or PCN phone connections 

5 61 r may also be provided. When the home system 258 is first initialized, the modem may be 

6 used to transfer the name and credit card information of the consumer to the billing and 

7 collection system 278 . The telephone connection 270 may be utilized each time an electronic 

8 book is purchased by a consumer to complete and record the transaction. The telephone 

9 connection 270 may also be used to receive the text data from the operations center 250, by- 

1 0 passing the video distribution system 208 . The phone connection 270 may be a separate unit 

1 1 as shown in Figure 6b. However, alternate means exist to connect the home system 258 to the 

1 2 billing and collection system 278 or the operation center 250. The modular connector 70 1 

1 3 (shown in Figures 6b and 8) provides access to each communication network to provide a path 

14 from the home system 258 to the billing and collection system 278 or the operations center 

15 250. These alternatives are presented in detail in Section VU. 

16 iv. Library Processing 

17 Figure 9 shows for one embodiment, an example of processing performed by the 

1 8 digital logic section 609 of the library 262 on the data stream 65 1 received from the video 

19 connector 2 12 or stripper circuit 617. In step S650, digital logic section 609 checks the data 

20 stream 65 1 for error correction . If an error is detected, in step S654 digital logic section 609 

2 1 de-interleaves the data and in step S65 8 runs a FEC (Forward Error Correcting) algorithm. 

22 In steps S650, S654 and S658, the digital logic section 609 performs the error correction 

23 needed on the data stream. If no error correction is necessary the digital logic section 609 

24 proceeds to step S662 and checks data packets individually for packet address. 

25 If the address is a unique address, the process moves to step S666 and the digital logic 

26 section 609 checks whether the address of the packet matches the library box ID number. The 

27 library box ID number is a unique number associated with the library 262. The library box ID 
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1 is used to ensure security of the data. The process then moves to step S670 and the digital 

2 logic section 609 determines whether an electronic file has already been opened into which the 

3 data packet can be saved. If no data file has been opened, the digital logic section 609 opens 

4 a new data file for that packet. If an electronic file has been opened, the process moves to step 

5 S678 and the digital logic section 609 saves the packet in the electronic file on disk. The 

6 process moves to step 682 and the digital logic section 609 checks to see if this is the last 

7 packet for a particular book for a particular textual data block being received. If it is the last 

8 packet of information, the process moves to step 686 and the digital logic section 609 closes 

9 the electronic file and updates the directory of available electronic files. Following either step 

10 S682 or S686, the process returns to receive another data packet from the data stream 

1 1 received from the data stripper block. 

12 If the packet address is checked and the address is determined to be a broadcast 

1 3 address, the process moves to step S690 and the digital logic section 609 determines the type 

14 of message that is being sent. The message may be an index of book titles, menu (and menu 

1 5 graphics) information, announcements, special offerings, discounts, promotions, and previews, 

1 6 for example. The process then moves to step S694 and the digital logic section 609 stores the 

17 message in an appropriate electronic message file. The process then returns to step S650 to 

18 receive another data packet and perform another error check. 

19 Using the process of Figure 9, the library 262 is able to receive, store and update 

20 directories related to the textual data and graphical data (that can be used to depict pictures 

21 in a given book or to generate menus). Variations of the processes are possible depending on 

22 the format of the data and operating system of the library 262. 

23 Figure 10 shows an example of the processing of information requests from the viewer 

24 266 at the library 262. Information requests from the viewer 266 are received either through 

25 the cable connecting the viewer 266 to the library 262 or through wireless transmissions such 

26 as RF. It is possible in some embodiments for subscribers 'requests to come from a set top 

27 converter box 602 (see Section V). 
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1 Moraiation requests received from the viewer 266 generally fall into three categories: 

2 (1) directory data of books stored in the library 262, (2) index of all available books on the 

3 system, and (3) requests for a specific book (step S700). In step S704, the digital logic section 

4 609 answers a request from the viewer 266 for a directory of data showing the books stored 

5 at the viewer 266. The directory of data is sent to the viewer 266 so that it may be displayed 

6 to the subscriber. In step S708, the digital logic section 609 handles requests from the viewer 

7 266 for an index of all available books on the system. The library 262 will obtain an index of 

8 all the available books on the system and transmit that index, in step S712, with menu 

9 information to the viewer 266. In step S7 1 6, the digital logic section 609 replies to a request 

1 0 from the viewer 266 for a specific book. In step S720, the digital logic section 609 opens an 

1 1 electronic file for the specific book requested by the viewer 266 and transmits the record or 

1 2 transmits the information on a packet-by-packet basis to the viewer 266. This process of 

1 3 transmitting the specific book, record, or packets to the viewer 266 continues until the last 

14 record or packet has been sent in step S724. 

15 In addition to the processes shown on Figure 1 0 in handling a request for a specific 

1 6 book, the library 262 also orders and receives specific books from the operations center 250 

1 7 using the process as described in step S7 1 6. Following a request for a specific book which 

18 is not stored at the Ubrary 262, the library 262 will proceed to determine the next available time 

1 9 the book will be on the video distribution system 208 or an altemative delivery system and 

20 ensure reception and storage of that book (process not shown), hi performing this process the 

2 1 library 262 will transmit to the viewer information on when it will obtain the text data for the 

22 book so that the subscriber may view the book. In addition to timing information, price and 

23 other ordering information may also be passed by the library 262 to the subscriber. 

24 c. The Viewer 

25 Figure 11 is a block diagram of a viewer 266 showing its internal components. The 

26 viewer 266 of Figure 1 1 is similar to the viewer 266 depicted in Figure 6b. The viewer 266 

27 is designed to physically resemble a bound book. The viewer 266 is made up of five primary 



-27- 



Docket 5281/PTO Filings/Spec wpd 

1 components and seven optional components: (1) LCD display 602, (2) digital circuitry (not 

2 shown), (3) video graphics contiroUer 607 ', (4) controls 740, (5) book memory 728, (6) 

3 optional powersupplycircuitiy 736, (7) optional battery 603', (8) optional RF transceiver 604, 

4 and (9) optional cellular or mobile connector (such as 611 ') (10) optional keyboards 267 and 

5 268, and (1 1) an optional speaker/microphone 608', (12) optional alternative communication 

6 interface devices. 

7 (1) AhighresolutionLCDscreen602,ofVGAquality,maybeusedbytheviewer 

8 266 to display text and graphic images. The screen may be the size of one page of an 

9 electronic book. A two page screen or two screens may also be used with the viewer 266. 

10 (2) Digital circuitrythatincludesasecureniicroprocessor621, instruction memory 

11 732, and digital logic. Data is transferred to the viewer 266 in compressed and encrypted 

1 2 format. The secure microprocessor 62 1 compares the JD number of the viewer 266 witii tiie 

1 3 incoming data sti-eam and only stores the text data if the ID number of the viewer 266 matches 

14 that within the incoming data stream. The viewer 266 may be configurcd to not output text data 

1 5 or other data and that the data is decompressed and decrypted only at tiie moment of viewing 

16 andonlyforthecunentpagebeingviewed. These measures provide additional security against 

17 unauthorized access to data. 

18 (3) A video graphics controller 607' that is capable of assisting and displaying 

1 9 VGA quality text and graphic images is included in the viewer 266. The graphics conti-oUer 

20 607 ' is controlled by the digital circuitiy described above. Text may be displayed in multiple 

21 font sizes. 

22 (4) The viewer 266 of Figure 1 1 has touch panel conti-ols 740. These unique and 

23 novel conti:ols 740 allow the consumer to select stored electtonic books and electi-onic books 

24 from catalogues, move a cursor, and turn pages in an electronic book. Typically, tiie controls 

25 740 include forward and reverse page buttons 742, 741, aball 743 for cursor movement, one 

26 or more selection buttons 745, a current book button 747 and a bookmark button 749 (see 

27 Figure 14a). 
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1 The controls 740 should be easy to use and conveniently located. Referring to Figure 

2 14a, the controls for the viewer 266 may be located below the screen 602 at the bottom 

3 portion of the viewer 266. The next page turn button 742 is the most used button 740 and is 

4 located towards the right edge of the page. The subscriber is likely to use right hand thumb 

5 movements to work the controls particularly the page tum buttons 74 1 , 742. Therefore, the 

6 buttons may be arranged in such a manner that the buttons are easily controlled by a 

7 subscriber' s right thumb. Generally, this can be accommodated either on the lower portion of 

8 the viewer 266 (as shown) or along the right hand margin of die viewer 266 (not shown). The 

9 current book button 747 and bookmark button 749 are usually the least used of the controls 

10 740. Therefore, in the example shown, those buttons 747, 749 are located on the inside 

1 1 portion towards the binder of the viewer 266Locating the ball 743 or other cursor 

1 2 movement device (such as four pointer arrows not shown) in the bottom center of the viewer 

13 266 is both easier for the subscriber to use aud easier in manufacturing the viewer 266. The 

14 selection buttons for the cursor 745 may be located below the middle diameter of the cursor 

1 5 ball 743 on the right and left sides of the ball as shown. If pointer arrows are used for cursor 

1 6 movement, a selection button 745 may be located in the center of the four arrow buttons (not 

1 7 shown) . Again, the most used controls 740 should be located where a subscriber' s right hand 

1 8 thumb would normally rest. 

1 9 (5) Book memory 728 for at least one electronic book or more of text is included 

20 in the viewer 266. The memory 728 stores text and any graphics, which represent pictures in 

2 1 a book. The memory 728 can also store menu graphics data. Two different memory 728 

22 devices may be used in the viewer 266, one for the instructions for the microprocessor 62 1 in 

23 the digital circuitry and a second type of memory may be used for the book memory 728 (and 

24 graphics). Various memory devices available on the market may be used such as, ROM, 

25 RAM or a small hard disk. Since an electronic book requires approximately 0.6 megabytes 

26 of storage, a small hard disk providing approximately 60 MBytes of storage provides memory 

27 to store approximately 100 electronic books. 
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1 Text for electronic books may be displayed in various font sizes. To accommodate 

2 various fonts for display, a variety of fonts are stored in instruction 732 or book memory 728. 

3 Thus larger or smaller fonts may be recalled from memory 62 1 , 728 to create displays desired 

4 by the subscriber, 

5 (6) Power supply circuitry 736 in the viewer 266 will accept power from either an 

6 AC power source or from an optional battery 603 ' , or the library 262. The power supply 

7 circuitry 736 provides the necessary voltages to accommodate the various systems within the 

8 viewer 266. 

9 (7) An optional battery 603 ' is provided in one embodiment. The battery 603 ' is 

10 automatically recharged when AC power is available. 

1 1 (8) An optional RF transceiver 604 which provided two-way data link between 

12 the viewer 266 and other components of the home subsystem can also be included in the 

13 viewer 266. 

14 (9) Also, the viewer 266 may include a cellular transceiver (not shown) for mobile 

15 communications. 

16 (10) The optional wired (attached) keyboard 267 and wireless (e.g., RF) keyboard 

1 7 268 (see Figure 6a) may be used with the viewer 266 to provide communications between the 

1 8 subscriber and the viewer 266 . 

19 (1 1) The optional speaker and microphone 608' allow the viewer 266 to provide 

20 audio signals to the subscriber, and allow the subscriber to provide an audio input. The 

2 1 speaker and microphone 608 ' may be used in conjunction with the cellular transceiver 608 or 

22 other telecommxuiications equipment to provide for reception and transmission of telephony and 

23 data. 

24 (12) The optional alternative communication interface devices allow the viewer 266 

25 to make use of a variety of communication paths. 

26 The viewer 266 of Figure 1 1 has parts available for providing connections to: a library 

27 744, electronic card memory 748, CD ROM units 752, and aportable memory unit 756 (such 
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1 as that shown in Figure 6b as 600'). Various electronic memory cards such as PCMCIA can 

2 be used with the viewer 266 to supply and store electronic books. 

3 Security, low power consumption and excellent display technology arc desired features 

4 ofthe viewer 266 design. The viewer 266 should be lightweight and portable. Theviewer266 

5 contains a software operating system that allows electronic books to be stored, read and 

6 erased and includes the capabiUty to order electronic books and retain them in memory 728 

7 for a predefined period of time determined by the system operator. The software can be 

8 configured to allow the electronic book to be read during a period of time (i.e., two weeks) and 

9 then automatic ally erased, read once and erased, or held in memory permanently. Each viewer 

1 0 266 may have a unique key 605 . All of the data storage may be encrypted with the key 605 

1 1 for an individual viewer 266 to prevent more than one viewer 266 accessing the text file or 

12 electronic book file. 

1 3 Figure 1 2 is a flow diagram of some of the processes executed by the microprocessor 

14 621 in the viewer 266. The viewer 266 may receive inputs from the subscriber through touch 

15 panel controls 740. In step S800, the subscriber's information requests are then processed by 

1 6 the microprocessor 62 1 . 

17 In step S804, if the subscriber requests a menu of available electronic books, the 

1 8 microprocessor 62 1 will select an electronic book menu. Li step S808, the microprocessor 

19 62 1 will open the electronic files that list the electronic books which are available (related to 

20 the category of topic of the menu) and display the menu with the names of the available 

2 1 electronic books . 

22 If the subscriber selects a particular book to read, then in step S812, the 

23 microprocessor 62 1 will process the selection and determine the electronic file that contains 

24 the specific electronic book. In step S8 16, the microprocessor 621 will open the file for that 

25 specific electronic book and normally access the first page. (If a pointer has already been set 

26 in that books electronic file, the process may default to that page.) In step S820, the 

27 microprocessor 621 will then determine which page needs to be displayed. That is, the 
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1 microprocessor 62 1 will determine whether a next page, previous page or a bookmarked page 

2 needs to be displayed. If the pointer for the electronic file is not in the correct location then in 

3 step S828, the microprocessor 62 1 will move the pointer and obtain the previous page of data 

4 from the stored file. Otherwise, in step S 824, the microprocessor 621 will normally obtain the 

5 next page of text from the stored electronic file. In step S832, the microprocessor 62 1 will 

6 decrypt and decompress the text data and send the data to the video display. The video 

7 display will generally have a video display memory associated with it. In step S832, the 

8 microprocessor 621 will send the data directly to that video display memory. The circuitry for 

9 the display then completes the process of displaying the page of text. 

10 If the subscriber, through the controls 740, requests (from step S800) that the power 

1 1 be turned off, then in step S836, the microprocessor 62 1 initiates power off. In step S840, the 

1 2 microprocessor 62 1 saves the pointer in memory to the page number in the book that the 

13 viewer 266 is currently reading. In step S844, the microprocessor 621 closes all the electronic 

1 4 files and signals the power circuitry to shut down the power to the various circuits in the viewer 

15 266. With these examples of basic processes the viewer 266 is able to display book selections 

16 and display text from those electronic books. 

17 d. Menu System 

1 8 Referring generally to Figure 1 3 , the electronic book system 200 may have a menu 

19 system 85 1 for selecting features and books from the electronic book system 200. The 

20 operating software and memory required for the menu system 851 may be located at the 

2 1 viewer 266 (e.g., the instruction memory 732 and/or book memory 728). However, it can also 

22 be located at the library 262 (e.g., the instruction memory 632) or the library 262 and the 

23 viewer 266 can share the software and memory needed to operate the menu system 85 1 . 

24 Since the menus are usually displayed on the viewer, and since the viewer 266 may be capable 

25 of operating in the absence of the library 262, the basic software and memory to create the 

26 menus is more conveniently located at the viewer 266. 
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1 The menu system 85 1 allows sequencing between menus and provides menu graphics 

2 for graphical displays such as on the LCD display 602 of the viewer 266. In an electronic 

3 book system that uses a set top converter these menus may also be displayed on a television 

4 screen. In an electronic book system that uses a computer, these menus may also be displayed 

5 on the computer monitor. In an embodiment, the menus provide just basic text information 

6 from which the subscriber makes choices . In other embodiments, the menus provide visual 

7 displays with graphics and icons to assist the subscriber and allow for subscriber interaction and 

8 real-time ordering of electronic books or other content available to the subscriber, 

9 Figure 1 3 depicts the menu system 85 1 with sequencing. The primary menus in the 

1 0 menu system 85 1 are an introductory menu 850, a main menu 854 and various submenus 858. 

11 In the embodiment shown, there are three levels of submenus 858. In certain instances one or 

1 2 two submenus 858 is sufficient to easily direct the subscriber to the selection or information 

1 3 requested. However, there are features in which three or more submenus 858 make the user 

14 interface more friendly for the subscriber. Each level of submenus 858 may consist of multiple 

1 5 possible menus for display. The particular menu displayed depends on the selection by the 

1 6 subscriber on the previous shown menu. An example of this tree sequence of one to many 

17 menus are the help submenus 887, 888. Depending upon the specific help requested, a 

18 different level two help menu is displayed to the subscriber. 

19 An example of an introductory menu 850 is shown on Figure 14a. Generally the 

20 introductory menu 850 introduces the viewer 266 to the system and provides initial guidance, 

2 1 announcements and instruction. The introductory menu 850 is followed by a main menu 854, 

22 an example of which is showninFigure 14b. Themainmenuprovides the viewer 266 with the 

23 basic selection or features available in the system. Figure 14b is an example of a main menu 

24 854 offering many additional features and submenus 858 to the subscriber. For example, 

25 Figure 1 4b shows that the viewer 266 is able to choose by a point and click method, many 

26 options including: ( 1 ) free previews, (2) books you can order, (3) books in your library, (4) 
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1 your current book, (5) help, (6) on-line services and (6) other system features. Following a 

2 selection on the main menu 854, a corresponding submenu 858 is shown. 

3 Figure 1 3 shows fourteen available primary or first level submenus. They are ( 1 ) 

4 account set up 862, (2) free previews 866, (3) book suggestion entries 855, (4) books in your 

5 library 872, (5) books you can order 878, (6) your current book 884, (7) help 887, (8) 

6 available features 890, (9) messages 893, (10) account information 896, (11) outgoing 

7 message submenu 898, (12) show links submenu 970, (13) create links submenu 980, and (14) 

8 show interactive files submenu 990. Figure 14c is an example of a first level submenu for 

9 books in your library 872. This "Book In Your Library" example submenu 872 shows six 

1 0 available books by title and author and provides the subscriber with the ability to check a 

1 1 different shelf of books 874 or return to the main menu 854. Figures 14d and 14e show 

12 example submenus 858 for books that may be ordered using the "Books You Can Order" 

13 submenu 878. 

1 4 Figure 14f is an example of a confirmation menu which confirms a subscribers order. 

15 In this particular example, the subscriber is required to enter a PIN number to complete the 

16 subscriber's order. Any alpha-numeric or similar password may be used to ensure the 

1 7 subscriber is an authorized subscriber. In one embodiment, the subscriber confirms an order 

1 8 with a PIN or password and then receives a final confirmation screen. The final confirmation 

19 screen is primarily text and may state: 

20 Your book order is now being processed using CABLE . 

2 1 Your book will be delivered overnight and your VISA account will be charged 

22 $2.95 . 

23 Your book will be available for reading at 6;00AM EST tomorrow. Make sure 

24 that: 

25 1. your Library Unit and Cable Connection Unit are plugged in with aerials up 

26 tonight; and 
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1 2. you tune your cable converter to THE BOOK ChanneL The TV set does 

2 not have to remain on. 

3 or similar language. 

4 Examples of the "Account Set Up Menu" 862 and further submenus 858 related to 

5 account set up (which provide instructions and account input 864) are shown in Figures 14g 

6 andFigure 14h. Thesesubmenus 858 allow initialization of an account at the operations center 

7 250 and orders to be charged to credit cards. The submenus 858 include the ability to enter 

8 data related to your desired PE^ number or password, credit cards, phone numbers, etc. In 

9 one embodiment, the account set up be performed using the telephone system. A confirmation 

1 0 menu verifies that the account has been properly set up with the desired PJN or password and 

1 1 credit card. However, additional set-up methods are presented in Section Vn. 

12 Free previews for books 866 are also provided by submenus (868, 870), Examples 

1 3 of the free preview menus are shown in Figure 14i and Figure 14j . Figure 14i shows a menu 

14 depicting various books for which previews are available for viewing. Following a book 

1 5 selection, a screen submenu showing an excerpt of the selected book cover's description is 

16 provided along with an excerpt from a critic's review of the selected book. In one 

1 7 embodiment, this preview screen for a particular book also allows the subscriber to select a 

1 8 submenu which provides information about the author. The book preview submenu may also 

1 9 include a still video picture or graphics portraying a book cover or a scene from the book. An 

20 example of such a still video picture or graphics is shown in figurc 14j which depicts a preview 

2 1 screen 870 about the author. The author' s preview screen 870 shows a picture of the author, 

22 provides a short biography, and may allow the subscriber to order the author' s books. The 

23 price for ordering the authors various books may also be shown on the menu. 

24 In addition to free previews, in other embodiments, the electronic book system 200 

25 provides the subscriber with abook suggestion feature (see 855). This is accomplished using 

26 the menu system 85 1 and the processor with associated memory located at the viewer 266, 

27 library 262 or at the distribution point (1020 or 250). When necessary, information for the 
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1 book suggestion feature is sent in the text data of the signal to the home system 258 . With this 

2 feature, books or authors are suggested to a subscriber based upon historical data of the 

3 subscriber' s previous orders, demographics or mood of the subscriber, other indicators, and/or 

4 by text word searches. 

5 In one book suggestion embodiment, text word searches of preview information (such 

6 as book cover descriptions, critics reviews and biographies about the author) and/or text of 

7 books or other titles are performed by the hbrary 262 using databases stored in the library 

8 memory 600. Personalized book or author suggestions are made to the subscriber by obtaining 

9 information from the subscriber indicative of general subscriber interests. Subscriber entries 

10 may be solicited from the subscriber using the book suggestion entry submenu 855. The 

1 1 system uses these subscriber entries either direcdy or indirectly to search for books or authors 

12 to suggest to the subscriber. 

1 3 Generally, the electronic book suggestion methods may be categorized into two 

14 categories, either responsive methods (which respond to a series of subscriber menu entries), 

15 or intelligent methods (which analyze data to suggest a book). Using a responsive or intelligent 

1 6 method, the system 200 determines a list of suggested titles or authors and creates a second 

17 or third level submenu 856, 857 to suggest the titles for subscriber selection. 

18 Responsive methods of suggesting titles include, for example, the use of mood 

1 9 questions, searching for authors, and keyword searching. Using die instruction memory 732 

20 and menu generation hardware (e.g., 607) of the viewer 266, a series of mood questions can 

21 be presented on menus to determine a subscribers interest at a particular time. For this 

22 methodology, the operations center' s 250 processor 404 and instruction memory 4 1 6 assign 

23 each title mood indicators (and sub-indicators) from a group such as light, serious, violent, 

24 short, long, dull, exciting, complex, easy-read, young theme, old theme, adventure, romance, 

25 drama, fiction, science-fiction, etc. These indicators are sent to the home system 258 with the 

26 text data and are stored in library memory 600. Based upon the subscriber entries, the 
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1 processor associates a set of indicators with the subscriber' s request and a set of books with 

2 matching indicators are located for suggesting to the subscriber. 

3 Responsive searches for authors or keywords (a search word provided by the 

4 subscriber) are generally performed by the hbrary processor 628 and instruction memory 632 

5 on data stored in the library memory 600, For example, a keyword given by the subscriber 

6 may be searched for a match in library memory 600 storing the book reviews, critics and 

7 previews databases. Thus, if a subscriber provided an entry of the word "submarine" on an 

8 appropriate submenu, the title "Hunt For Red October" may be located by the microprocessor 

9 628 using instruction from a routine in instruction memory 632. 

10 Intelligent methods of suggesting programs include analyzing personal profile data on 

1 1 the subscriber and/or historical data about the subscriber such as past books ordered by the 

12 subscriber (or buy data). This method maybe performed at the distribution point or operations 

1 3 center 250 by the on-site processor 404 using subscriber databases stored in memory 428 . 

1 4 The home system 258 receives the text data including program suggestion information from the 

1 5 distribution point or operations center 250 and generates the program suggestion submenus 

16 855, 856, 857 using the same text data receiving 2 1 2 and viewer menu generation hardware 

17 (e.g., 607, 621) described above. Software routines and algorithms stored in instruction 

18 memories (e.g. 632, 732) are used to analyze historical data and book ordered data to 

19 determine a line of books to suggest to the subscriber. 

20 The algorithms for this powerful feature of suggesting books or authors to subscribers 

21 is disclosed in great detail in U.S. Patent Number 5,798,785, entitled TERMINAL FOR 

22 SUGGESTING PROGRAMS OFFERED ON A TELEVISION PROGRAM DELIVERY SYSTEM, filed 

23 December 2, 1993, which is incorporated herein by reference. 

24 Referring to Figure 13, submenus 858 are shown on the "Books In Your Library" 

25 submenu 872 and may be broken into shelf numbers with submenus for each shelf 874, 876. 

26 The submenus 858 for the "Books You Can Order" submenu 878 is similarly broken out into 

27 submenus by shelves 880, 882. These shelves may each be a category or genre of books. 
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1 Books may be grouped into categories such as best sellers, novels, fiction, romance, etc. See 

2 Figure 14d. 

3 Referring to Figure 1 3, the submenu 858 for "Your Current Book" 884 allows a 

4 subscriber to select a current book 884 and then determine what page to view. This selection 

5 is confirmed with a level two submenu 885, The help submenu 887 provides the subscriber 

6 with additional help screens 888. The submenus 858 for available features 890 may be broken 

7 out into a sequence of separate submenus for each feature 891, 892. 

8 Referring to Figure 1 3 , messages can also be sent with the electronic book selection 

9 and delivery system 200. A level one message screen provides the subscriber with the ability 

1 0 to select from various messages the subscriber has pending 893 . Each message is then shown 

11 on a separate submenu screen 894, 895. The message may contain text and graphics. 

1 2 Referring to Figure 1 3 , account information is shown on a level one submenu 896 and 

1 3 then follow-on submenus 858 show the recent orders and your account balance 897, There 

14 is also a level one submenu for outgoing messages 898 which has a follow-on submenu used 

15 as an input screen 899. 

16 In addition to the specific features and submenus described in Figure 1 3 and Figure 14a 

17 throughFigure 14j, many other variations andfeatures are possible. When a book is finally 

1 8 selected for viewing the title page 886 will appear on the screen followed by a page of text. 

19 m. Billing And Collection System 

20 In one embodiment, the billing and collection system 278 (shown in Figures 2 and 3) 

2 1 utilizes the latest technology in electronic transaction and telephone switching to track orders, 

22 authorize dehveries, bill consumers, and credit pubUshers automatically. The telephone calls 

23 initiated by the phone connector 270 are received by the billing and collection system 278 

24 which responds immediately without human intervention by placing the order and charging the 

25 consumers credit card account. Data is compiled periodically and publishers 282 are credited 

26 for sales of their books or other text. The billing and collection system 278 may also connect 
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1 with subscribers through two-way cable connections, cellular, or other communication means. 

2 These additional methods are detailed in Section Vn. 

3 The billing and collection system 278 communicates with the operations center to track 

4 changes in available books and to provide statistical data to the operations center 250, 

5 IV. Public Library, School, and Bookstore System 

6 The electronic book system can be modified to be used at public libraries, schools, 

7 bookstores, newsstands, or stand-alone kiosks. Figure 1 5 shows one possible arrangement 

8 of components for the distribution location. The main unit is the file server 900. The file server 

9 900 is a large electronic memory unit that can store thousands of books, newspapers, or 

10 periodicals. Various electronic storage means maybe used in the file servers, such as hard 

1 1 disks, read-write CD ROMs and read-only CD ROMs. 

12 The system comprises five components; the file server 900, a converter or video 

1 3 connector 904 or connector capable of interfacing to one of the alternative delivery systems 

14 presented in Section VII, a controller 908, a viewer 912, and a catalog printer 916. The 

1 5 software for controlling the system is primarily located in the controller 908 . The converter or 

1 6 video connector 904 is sinoilar to those described above. In this configuration the controller unit 

17 908 monitors the data being transferred to the file server 900 by the converter 904. The 

1 8 controller 908 may be provided with a viewing screen and several control buttons. When it 

19 is necessary to have a larger screen to perform more sophisticated controlling of the system a 

20 viewer 266 may be connected to the controller 908 and the viewer screen and controls 740 

21 may be used. 

22 For security reasons, the controller 908 is only able to download books to public 

23 viewers 9 1 2 which are authorized to receive books from the particular file server 900. Also 

24 for security reasons it is not desirable that the public viewer 9 1 2 have access to more than one 

25 file server 900. Li this way, security can be maintained over the text data for books. The 

26 public viewer 9 1 2 may be limited to receiving one or two books at a time from the controller 

27 908. Whentheuserof thepubhc viewer912needs a new or additional book, the user returns 
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1 the viewer 9 1 2 to the school or pubUc library where the user receives a new book from the 

2 controller 908. 

3 In order to track the books that are available on the file server 900, the titles of the 

4 available books may be printed on a catalog printer 9 1 6. The catalog printer 9 1 6 is connected 

5 to the library controller 908 and the titles of the books are downloaded to the catalog printer 

6 916. For security reasons, the coded text for any of the electronic books may not be 

7 authorized for printing using the controller 908 and catalog printer 916. In order to maintain 

8 security over the data, none of the electronic book data may be allowed to be downloaded to 

9 the printer 916. Once a complete printout of available book titles, magazines, or other textual 

1 0 material is complete, a hard copy of the catalog 920 can be maintained at the file server 900. 

1 1 The system shown may also be used at bookstores. The bookstores can rent the 

1 2 public viewer 9 1 2 to customers with the text for one or two books loaded onto the public 

13 viewer912. The public viewer 9 12 may be provided with an automatic timeout sequence. The 

1 4 timeout sequence would erase the textual data for the books after a certain period of time, for 

1 5 example, two weeks. It is expected that after a period of time (perhaps within two weeks) the 

1 6 renter would return the public viewer 9 1 2 to the bookstore and receive additional books for 

17 viewing. Using this arrangement, it is also possible for the bookstore to (permanently) sell a 

1 8 viewer 9 1 2 to a regular customer. The customer then returns to the bookstore from time to 

1 9 time to receive textual data for a book which the customer can then store permanently on the 

20 customer' s own viewer 9 1 2. Various other configurations are possible for bookstores, schools 

21 and public libraries using the file server 900 and public viewer 912 described. 

22 V. Use Of A Set Top Converter 

23 Existing set top converter boxes such as those made by Scientific Adanta or General 

24 Instruments are presently unequipped to handle the book selection system of the present 

25 invention. Although set top converters may be built which include the library functions, 

26 hardware modifications are necessary in order to use the book selection system with existing 

27 set top converter technology. 
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1 Figures 16a and 16b are examples of hardware modifications or upgrades. Aport is 

2 used to attach hardware upgrades described below to a set top terminal. Two upgrades are 

3 possible to set top converters 601 to assist in receiving and selecting electronic books, amenu 

4 generation card upgrade (Figure 16a) and an information download unit (Figure 16b). Each 

5 of these upgrades may be connected to the set top terminal unit through an upgrade port. A 

6 four wire cable, ribbon cable, FireWire (IEEE 1 394B) interface connector, USB connector, 

7 or the like may be used to connect the upgrade to the set top converter 601. 

8 A card addition 950 to a set top converter 60 1 is depicted in Figure 1 6a. The card 

9 950 shown provides the additional functionaUty needed to utiHze the book selection system with 

10 existingsettopconverter601 technology. The card 950 may be configured to slip inside the 

1 1 frame of a set top terminal and become part of the set top terminal, an advanced set top 

12 terminal. The primary functions the card 950 adds to the set top converter 601 are the 

1 3 interpreting of data signals, generating of menus, sequencing of menus, and, ultimately, the 

14 abihty of the subscriber to select abook using either the television or a viewer 266. The card 

15 950 also provides a method for a remote location, such as the cable headend, to receive 

1 6 information on books ordered. The books ordered information and control commands may 

17 be passed from the cable headend to the card 950 using telephone lines or alternative ordering 

18 methods as presented in Section VU. 

1 9 The primary components of the card 950 are a PC chip CPU 952, a VGA graphic 

20 controller 954, a video combiner 956, logic circuitry 958, NTSC encoder 960, a receiver 962, 

21 demodulator (not shown), and a connector 61 1 which consists of a dialer. The card 950 

22 operates by receiving the data text signal from the cable headend through the coaxial cable. 

23 The logic circuitry 958 of the card 950 receives data 964, infrared commands 966, and 

24 synchronization signals (not shown) from the set top converter 601 . Menu selections made by 

25 the viewer 266 on the remote control are received by the set top converter' s 60 IIR equipment 

26 and passed through to the card 950. The card 950 interprets the IR signal and determines the 

27 book (or menu) the subscriber has selected. The card 950 modifies the IR command to send 
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1 the information to the set top converter 60 1 . The modified IR command contains the channel 

2 information needed by the set top converter 60 1 . Using the phone line 968 and dialer 611', 

3 the card 950 is able to transmit electronic books ordered information to the cable headend. 

4 It is also possible to receive the electronic books over the telephone lines and other 

5 telecommunications networks, including wireless networks, and by-pass the video distribution 

6 system. 

7 These commands are passed through the interface linking the set top terminal's 

8 microprocessor with the microprocessor of the hardware upgrades. In this way, subscriber 

9 inputs, entered through the set top terminal keypad or remote control, can be transferred to any 

10 of the hardware upgrades for processing and responses generated therein can then be sent 

1 1 back to the set top terminal for display. In one embodiment the IR commands 966 are 

12 transferred from set top terminal 601 to hardware upgrade. 

1 3 Hardware upgrades may include a microprocessor, interactive software, processing 

14 circuitry, bubble memory, and a long-term memory device. In addition to these basic 

1 5 components, the hardware upgrade may make use of an additional telephone modem or CD- 

16 ROM device. 

17 An information download hardware upgrade 1001 showninFigure 16b allows the 

1 8 subscriber to download large volumes of information from the operations center 250 or cable 

19 headendusingasettopterminal610. The hardware upgrade 1001 will enable subscribers to 

20 download data, such as electronic books and electronic magazines, to local storage. Primarily, 

2 1 the hardware upgrade 1001 is an additional local storage unit 1003 (e.g., hard disk, floppy, 

22 optical disk or magnetic cartridge and may include a microprocessor 1 005 , instruction memory 

23 1007, and a random access memory 1009, as shown in Figure 16b). A small portable viewer 

24 may also provided with the upgrade 1001 to enable downloaded text to be read without the 

25 use of a television. 

26 The downloadable information may be text or graphics supplied by the operations 

27 center 250 or cable headend. With tiie upgrade 100 1 , electronic books may be downloaded 
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1 and read anywhere with the viewer 266. Using the upgrade 1001, electronic books maybe 

2 downloaded and stored in compressed form for later decompression. The electronic books 

3 may be decompressed only at the time of viewing. Important text that the public desires 

4 immediate access may made available through this system. Text such as the President's 

5 speech, a new law, or a recent abortion decision rendered by the Supreme Court may be made 

6 immediately available. 

7 In one embodiment, electronic book ordering information is stored at each set top 

8 terminal 610 until it is polled by the cable headend using a polling request message format. An 

9 example of a polling request message format consists of six fields, namely: ( 1 ) a leading flag 

10 at the beginning of the message, (2) an address field, (3) a subscriber region designation, (4) 

11 a set top terminal identifier that includes a polling command/response (or P/F) bit, (5) an 

1 2 information field, and (6) a trailing flag at the end of the message. A similar response frame 

1 3 format for information communicated by the set top terminal to the cable headend in response 

14 to the polling request may be used. 

15 Figure 17showscomponentsofasettopterminal610'. The components include a 

16 datareceiver617' and a data transmitter 101 1. The data transmitter provides upstream data 

17 communicationscapabilitybetweenthesettopteraiinal610'andthecableheadend. Upstream 

1 8 data transmissions are accomphshed using the polling system described and, using a data 

19 transmitter 101 1 . Both receiver 617' and transmitter 101 1 may be built into the set top 

20 terminal 610' itself or added through an upgrade module. Regardless of the specific hardware 

2 1 configuration, the set top terminal' s data transmission capabilities may be accomplished using 

22 the hardware shown in Figure 17. 

23 Figure 17 shows RF signals, depicted as being received by a data receiver 6 17 ' and 

24 tuner613 working in unison. Both of these devices are interfaced with the microprocessor 

25 1013, which receives inputs 1015, from the subscriber, either through a set top terminal's 

26 keypad, a remote control unit or viewer 266. Generally, all cable signals intended for reception 

27 on the subscriber's TV are accessed by the tuner 61 3 and subsequently processed by the 



-43- 



Docket 5281/PTO Fifmgs/Spec wpd 

1 processing circuitry 1017. This processing circuitry 1017 typically includes additional 

2 components (not shown) for descrambling, demodulation, volume control and remodulation on 

3 a Channel 3 or 4 TV carrier. 

4 Data targeted to individual set top terminals is received by the data receiver 617' 

5 according to each set top terminal' s specific address or ID. In this way, each addressable set 

6 top terminal 610' only receives its own data. The data receiver 617' may receive set top 

7 terminal 610' specific data in the information field of the signal frame described or on a separate 

8 data carrier located at a convenient frequency in the incoming spectrum. 

9 The received data includes information regarding electronic books and menus available 

1 0 for selection. The subscriber may enter a series of commands 1015 using a keypad or remote 

1 1 control in order to choose an electronic book or menu. Upon receipt of such commands, the 

1 2 set top terminal' s microprocessor 1013 instructs the tuner to tune to the proper frequency of 

13 the channel carrying data and subsequently instructs the processing circuitry 1 0 1 7 to begin 

14 descrambling of this data. 

1 5 Upon selection of an electronic book, the microprocessor 1013 stores any selection 

1 6 information in local memoiy (not shown) for later data transmission back to the cable headend. 

1 7 The set top terminal' s microprocessor 1013 coordinates all CATV signal reception and also 

1 8 interacts with various upstream data transmission components. Typically, the data transmitter 

19 101 1 operates in the return frequency band between 5 and 30 MHZ. In an alternative 

20 embodiment, the frequency band of 10 to 1 5 MHZ may be used. Regardless, however, of the 

2 1 fi-equency band used, the data transmitter 1011 sends information to the cable headend in the 

22 information field of the response firame described. Those skilled in the art will recognize that 

23 a number of variations and combinations of the above-described set top terminal hardware 

24 components may be used to accomplish upstream data transmissions. 
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1 VL Books-On-Demand System 

2 The electronic book system 200 described may also be configured in a book-on- 

3 demand style. Figure 1 8a shows one example of a configuration for a book-on-demand 

4 system. A book on demand systemrequires more powerful two-way communications between 

5 the consumer's home, bookstore, school or public library and either the operations center 250 

6 or a distribution site 1020 such as the cable headend. In one embodiment, this type of two- 

7 way communication can be provided by the hardware shown in Figure 17 and described 

8 above. Additional methods related to alternative communication paths are presented in Section 

9 vn. 

1 0 Referring to Figure 1 8a, in a book-on-demand system, the subscriber selects the book 

11 to be download from an available menu ofbooks (see for example Figures 14dand 14e). The 

1 2 data for menus of available books is usually sent to the subscriber location by the distribution 

13 site 1020. After the subscriber's menu selection, information about the subscriber selection (or 

14 request) is then communicated to either a distribution point 1020 (such as a cable headend) or 

1 5 the operations center 250. Upon receipt of this request, the needed textual and graphical 

1 6 information for the book is spooled and sent to the subscriber. In this manner, electronic books 

1 7 are only sent when requested by the subscriber and are sent immediately upon demand for the 

1 8 electronic book (or text). 

1 9 In order to support such a demand system, the text delivery and distribution must be 

20 conducted on a strong nodal architecture distribution system, such as, a video-on-demand 

2 1 cable or telephone television system, through use of individual telephone calls on the public 

22 telephone system or cellular phone system, through the use of the Intemet, or a number of other 

23 data network options. 

24 The book-on-demand system allows for a greater selection of electronic books to the 

25 subscriber and hmits the amount of communicated book data that is unnecessary or unneeded. 

26 It also provides the electronic book to the subscriber in a much timelier fashion. 
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1 In addition to a stronger distribution system, a book-on-demand system requires a 

2 distribution point 1020 to have more sophisticated equipment to access and "spool out" the 

3 textual information. This can be accomplished using file server technology 1024 for storing the 

4 books and distribution technology such as ATM 1028 or telephone-type switching (not shown) 

5 to distribute the texUial information. The file server 1024 and distribution technology that can 

6 be used in configuring such a book-on-demand system is described in U.S. Patent No. 

7 5,262,875 and U.S. Patent 5,218,695, cited above. 

8 Figure 18a shows an embodiment for a book-on-demand system that utilizes file server 

9 technology. In addition to books, the embodiment of Figure 1 8a will support distribution of 

10 nearly any digital data. Books or textual files are received from publishers 282 and other 

1 1 sources through local feeds 1032, ATM 1028, orby satellite dish 1036, for example. Thedata 

1 2 is then stored in memory 1040 at the file server 1024. In one embodiment, the distribution 

1 3 point 1 020 is a cable headend that receives requests from subscribers and delivers text to 

14 subscribers over a two-way communication system (such as a video-on-demand system 

15 (VOD) 1044). 

1 6 The library 262 can be connected to either a basic premium-type service cable system 

17 1048, a near video-on-demand type cable system (or pay-per-view (PPV) 1052) or a video- 

18 on-demandcable system 1044. In connecting with either ofthese three systems the library 262 

19 may access the cable directly or may access the system through a set top terminal 601 ', 601 ", 

20 or 601"'. 

2 1 Using the two-way video-on-demand system 1044, a subscriber is able to request a 

22 specific book titie and receive that text immediately following its request. To accomplish this, 

23 the distribution point 1020 ti:ansmits a list of available books through the cable delivery system 

24 to tiie library 262. The library 262 displays tiie list of available books on a menu or similar 

25 format. As described earlier, the library 262 may use menus which list categories of available 

26 books to form its request from die distiibution point 1 020. After selecting a book the library 

27 262 then sends a request signal on the two-way communication system 1 044 back to the 
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1 distribution point 1020. This request signal can be handled in two ways. The library 262 either 

2 initiates the request or the disOibution point 1 020 polls the various libraries on to the two-way 

3 system 1044. Upon receiving the request for the book tide, the text associated with that book 

4 title is ti-ansmitted to the library 262 using the two-way cable system 1044. 

5 Figure 1 8b is an expanded view of an operations center 250 that supports a regional 

6 or national book-on-demand system. Li fact, tiie operations center 250 shown supports 

7 distiibution of nearly any digital data. The operations center 250 supports multiple feeds to 

8 receive digital information by tape 1060, 1060', ATM 1028, or satellite 1036. The information 

9 is processed through an input MUX 1064 and a small file server 1068 before reaching the 

10 master ffle server 1072. Digital data such as books received from publishers 282 is then stored 

1 1 on the master file server 1 072. The digital data may be stored compressed in a standard 

1 2 format such as MPEG2. 

1 3 A system controller 1 076 provides conti-ol over the regional or national book-on- 

14 demand system. Books may be packaged into groups to provide feeds to various cable 

1 5 headends. In addition, scheduling and marketing research are conducted at the operations 

1 6 center 250. In order to handle the scheduling and market research, book buy data is received 

17 at the operations center 250 through a multiplexer 1082. Book buy information can be 

1 8 provided by the operation center 250 to the billing and collection system 278. 

1 9 The operations center 250 is also equipped to insert messages or advertisements into 

20 the file server. These messages or advertisements will eventually be received by the 

21 subscribers. 

22 The master file server 1072 uses an output multiplexer 1080 and ATM 1028 as well 

23 as satelUte connections to distribute digital data. In one embodiment, cable headends receive 

24 text data on books from the master file server 1080 through the output multiplexer 1028 and 

25 an ATM system 1028. After receiving the digital book data, the cable headends store the 

26 books in a local file server 1024. Figure 18a's distribution point 1020isanexampleofacable 
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1 headend which may receive data from the operations center 250 of Figure 18b through an 

2 ATM hookup 1088 or sateUite hookup. 

3 Vn. Alternative Delivery And Ordering Methods 

4 Electronic books and related data, including electronic book menu data, may be 

5 provided to subscribers by use of an on-demand delivery system in which electronic books are 

6 delivered after an order is received by the delivery system. The delivery system may supply 

7 the electronic books in real time or near-real time (i.e., near on-demand), or after a delay 

8 period that allows the delivery system to process, package and transmit the electronic book. 

9 Alternatively, the delivery system may broadcast one or more electronic books in a continuous 

1 0 fashion. In this alternative, the subscriber indicates a desired electronic book from a list of the 

1 1 broadcast electronic books. The deUvery system may include abilling system that debits a 

1 2 subscriber's account, or debits acreditcard, for example, upon delivery of the electronic book. 

1 3 The delivery system, or arelated authorization system, may provide a local authorization code 

1 4 that allows the subscriber to decrypt, store and view the desired electronic book. Alternative 

15 delivery systems and methods are disclosed in pending U.S. Apphcation Serial No. 

16 09/289,956entitied,EI£CrRONICBOOKALTERNATIVEDELIVERYMETHODS,filed 

17 AprH 13, 1999, and U.S. Apphcation Serial No. 09/427,938 entitled, VIRTUAL ON- 

18 DEMAND ELECTRONIC BOOK, filed October 27, 1999, the disclosure of which are 

19 incorporated by reference. 

20 a. Internet Delivery Methods 

21 Figure 1 9 is an altemate delivery plan 301' that provides for electronic book delivery 

22 using the Internet. In Figure 26, the publishers 282 provide the electronic books to be posted 

23 at the Internet web site 279. The publishers may convert the text and graphical data to digital 

24 format, compress the digital data, and upload the compressed digital data to the Internet web 

25 site 279. Alternately, the publishers 282 may arrange for an outside conversion activity 283 

26 to convert the text and graphical data to digital format. The conversion activity 283 may then 

27 provide the digital data to the Internet web site 279 . For example, a large on-line bookstore 
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1 could gatherpublications in electronic formfroma variety of publishers, or could convert hard- 

2 copy books to electronic form, and post the electronic books on the Internet such as at the 

3 hitemet web site 279. 

4 The electronic books may then be transferred using a public switched telephone 

5 network (PSTN), for example, or other conmiunications systems, direct to a subscriber 285, 

6 a library 286 and a bookstore 287. The library 286 and the bookstore 287 may also provide 

7 electronic books to the subscriber 285. 

8 When electronic books are provided by the hitemet web site 279, the billing and 

9 collecting functions may be incorporated into the hitemet web site 279. For example, a 

10 subscriber may pay for an electronic book selection by entering a credit card number into a 

1 1 data field of a page of tiie Litemet web site 279. In this configuration, a separate biUing and 

12 collection system may not be required. Alternatively, the Litemet web site 279 may 

13 communicate information with the billing and collection system 278. 

14 Electronic book dehvery over the hitemet maybe handled using a number of methods. 

15 In a method, the electronic book may be downloaded to the requesting home system 258 

1 6 immediately after the order has been processed. Altemati vely , the electronic book may be e- 

17 mailed to an e-mail address that is entered as part of the ordering process. In another 

1 8 embodiment, as part of the ordering transaction process, the subscriber is provided with 

19 location and authorization information that allows the subscriber to retrieve the ordered 

20 electronic book at the subscriber's convenience. For delivery of subscription electronic book 

21 products, like newspapers, magazines or other periodicals, the Internet web site 279 can 

22 dehver the latest version of the product to the subscriber automatically immediately upon logon 

23 by the subscriber to the hitemet web site 279. Electronic book data may also be embedded 

24 into continuous multicast streaming video, audio, or data feeds. 

25 Dehvery methods that allow for dedicated, full time delivery can be used to provide 

26 continuous distribution of electronic book data including requested electronic books from 

27 subscribers, electronic books to be broadcast to all subscribers, updated menu contents, and 
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1 updated advertising. For delivery methods that allow for non-dedicated or user established 

2 connectivity, the operations center 250 may manage the timing and delivery of content by 

3 delivering the electronic book only when requested, or periodically, to ensure delivery for those 

4 home systems that may not be able to receive the delivery. Finally, delivery methods that are 

5 capable of two-way communication may be used to provide a return path to the operations 

6 center 250 or billing and collection system 278 for the purpose of ordering or requesting 

7 updated electronic book information. Two-way communications paths maybe used to allow 

8 linking from an electronic book home system 258 or viewer 266 to extemal nodes that provide 

9 world watch live content. These delivery methods are described in detail in co-pending U.S . 

10 Patent Application Serial No. 09/289,956 entitled ELECTRONIC BOOK ALTERNATIVE 

1 1 DELIVERYMETHODS, filed April 13, 1999, the disclosure of which is hereby incorporated 

12 by reference. 

1 3 Additionally, although the home system 258 is presented as an independent device that 

14 directly interfaces with the delivery system 200, the home system 258 may connect to the 

1 5 delivery system 200 through a set top terminal, TV, PC, radio, or any other device capable of 

1 6 receiving the signal provided by the delivery system 200. Additionally, the home system 258 

17 may not be an independent apparatus, having some or all of its functionahty supported within 

1 8 the set top terminal, TV, PC, radio, or any other device capable of receiving the signal 

19 provided by the delivery system. 

20 b. Alternative Ordering Methods 

2 1 Any of the delivery methods described in the section above are viable embodiments 

22 for providing access from the home system 258 to the billing and collection system 278 or 

23 operations center 250. In an embodiment, the modular connector 70 1 , as shown in Figures 

24 6b and 8, may be used to provide the specific protocol formatting and transmission processing 

25 to allow the home system 258 to use the communication path. In the embodiment where the 

26 PSTN is used to provide access, the modular connector 701 includes the phone connector 270 

27 and the modem 6 1 1 as depicted in Figure 6b. In the embodiment where the cellular phone 
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1 system is used to provide access, the modular connector 70 1 includes the cellular phone or 

2 PCN phone 6 1 T as depicted in Figure 6b. 

3 c. Mobile Environments 

4 A feature provided by deUvery methods using a wireless broadcast system, satellite 

5 broadcast system, v^ireless personal communication system, or terrestrial television broadcast 

6 system is mobility of the electronic book home system 258 or viewer 266. This mobility allows 

7 for ordering and receiving electronic book data anytime or anywhere, from sitting on abeach 

8 in Florida to sitting on a bus in New York City. This mobility allows for the delivery of 

9 electronic book data subscription products such as daily newspapers, monthly magazines, or 

10 books from book-of-the-month clubs. These subscription products may be delivered 

1 1 automatically to the appropriate home system 258 or electronic book viewer 266. Enhanced 

1 2 end-to-end error correction techniques can be added to the transmission system to ensure 

1 3 higher probability of receipt for these mobile environments. Additionally, transmission methods 

1 4 may be implemented that resend packets of electronic book data, changing their delivered 

15 order on each resend, to improve likelihood of receipt. For lower bandwidth mobile 

1 6 environments , an electronic book file may be broken up into packets and the packets sent a 

17 limited number of times. Ifthe electronic book file is notreceived completely, the electronic 

1 8 book viewer 266 may initiate a request to the operations center 250 to resend only that portion 

19 of the electronic book file yet to be received. 

20 Vin. Electronic Book Link System 

2 1 Electronic book links allow the subscriber to use the electronic book viewer 266 to 

22 traverse pre-defined paths between content in their currently viewed electronic book to related 

23 information contained either elsewhere in the electronic book, elsewhere on the viewer 266, 

24 or extemal to the viewer 266, including in the library unit 262 or in a connected Internet web 

25 site. These links provide an organized and methodical method for the subscriber to quickly 

26 access related topic areas or seek clarification of the currently viewed material. 
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1 An electronic book includes first locations, or components, such as words, phrases, 

2 sentences, sections of text, paragraphs, pages, chapters, figures, drawings, maps, video clips, 

3 and audio clips. Links to second and subsequent locations, or components, contained in the 

4 same electronic document or in another related electronic document, file, or database can be 

5 associated with each of these fu-st components. First components with underlying links can be 

6 highlighted and displayed on the viewer display 602 or on the connected television 259 or a 

7 personal computer 261 (see Figure 2). First components that have underlying hnks associated 

8 withthemmaybeidentifiedbyassigningthemauniqueidentifier. The unique identifier can be 

9 a word or phrase, an alpha-numeric value, a coordinate point, or other unique identifier. In an 

10 embodiment, each such first location may be assigned an identifying index value. 

1 1 The use of the index value allows the first components to maintain links with other 

1 2 components, even if the electronic book is altered. For example, a subscriber may use a cut 

1 3 and paste edit feature to move a block of text containing a first component. Cutting and pasting 

14 will not affect the stams of the first component and its links to other components . Similarly, 

15 changing font style or font size will not affect the status of the links. 

16 In the creation of an electronic book, or subsequently, the electronic book may 

17 undergo a process that maps identifying index values to each of the first components. These 

18 index values can then be accessed by software directives that drive the presentation of 

1 9 alternative or linked material (e.g. , material at one of the second locations) once a selection is 

20 made. For each electi-onic book, these index values may be contained in a hidden table that 

2 1 maps the identifying index values of aU first components with underlying links to the location of 

22 the linked material (the second location). Moreover, each such first component may be hnked 

23 to one or many linked material locations. That is, the first component may be linked to a 

24 second component, a third component and so on. An example of a hidden table is presented 

25 below. 
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Identifying 
Index 
Value(s) 


Component 
Identifier 


Link 
Number 


Linked Material 
Identifier 


Linked Material 
Description 


Linked Material 
Location (file location / 
file name / 
corresponding index 
value) 


135 


"Cezzanne" 


1 


More on 
Cezzanne 


Reference material on 
Cezzanne 


Art- 

Encyclopedia.com/Fren 
chArtists/Index Value = 
1 


135 


"Cezzanne" 


2 


Pronunciation 


Pronunciation of tiie 
word 


Websters.com/ 
Websters E- 
Dictionary/Index Value 
= 56221 


135 


"Cezzanne*' 


3 


Audio Clip 


Audio file providing 
condensed Cezzanne* s 
biography 


Viewer/ 
Current file/Index Value 
= 199384 


133-135 


"PorchScene 
by Cezzanne" 


1 


Graphic File 


JPEG file presenting 
Cezzanne' s PorchScene 
painting 


Viewer/ 
Current file/Index Value 
= 9382 


5673 


"reactivism" 


1 


Definition 


Definition of the word 


Websters.com/ 
Websters E- 
Dictionary/Index Value 
= 564 


4948- 
4950 


"Order little 
Women" 
menu item 


1 


Book Order 


Order the book Littie 
Women 


Discovery.com/ 
Little Women 
Order/Index Value = 
672 


4949- 
4950 


"Little 
Women" 


1 


Book review 


Review of the book 
"Little Women" 


LiteraryWorks.com/ 
Little Women/hidex 
Value = 1 


90462 


"Dental 
diseases" 


1 


TOC Unk to 
Document Body 


Link from Table of 
Contents to desired 
chapter 


Viewer/ 
Current file/Index Value 
= 69980 


1342 


"Dental 
diseases" 


2 


Related 
discussion group 


Access to Web discussion 
group on gum diseases 


NoMoreCavitiesxom/ 
Index Value = 1 


572 


"VegieMaster" 


1 


Product Order 


Order the product 
^'VegieMaster" 


HomePurchases.com/ 
KitchenProducts / 
Index Value = 1 


14 


"Chesapeake" 


1 


Video 


Video clip of interview 
with L Michener on 
writing of Chesapeake 


Viewer/ 
Current file/Index Value 
= 38677 


14 


"Chesapeake" 


2 


Narration 


Audio file - narration of 
Chesapeake by J. 
Michener 


Viewer/ 
Current file/Index Value 
= 38678 



As shown in the table, a first location "Cezzanne" has an identifying index value 135. 
"Cezzanne" has three links. A link to a second location is to an art encyclopedia. A link to a 
third location is to an electronic dictionary that provides a pronunciation guide. A link to a 
fourth location is to an audio file that plays a short biography of the artist. Each of the second, 
third and fourth locations have their own index valuei^inked material location information 
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1 (i.e., the location of second and subsequent components) can include source location, book 

2 name, chapter, page, line, and word as identified by their index value. The source location will 

3 provide the delivery system 200 the necessary information to contact the operations center 

4 250, the hitemet web site 279 (see Figure 2) or another electronic database and request the 

5 appropriate material for retrieval and download if it currendy does not reside on the viewer 266 

6 or the home system 258. Li the case that the hnked material resides on the Internet web site 

7 279 or on another electronic database, the location information in the hidden table allows the 

8 operations center 250 or home system 258 to retrieve the desired material fi-om the Internet 

9 web site 279 or firom the electronic database. 

1 0 If the second component, or linked material, is located at the viewer 266, the processor 

1 1 in the viewer 266 can cause the hnked material to be displayed without any communications 

1 2 with an outside source. For example, if the first component is the name "Cezzanne" and the 

1 3 hnked material, or second location, is in an electronic dictionary stored in the viewer 266, the 

14 viewer 266 can display the electronic dictionary entry for "Cezzanne." This hnked material may 

1 5 be displayed full screen, in a picture-in-picture window, or as an overiay, for example. The 

16 entry can also remain hidden, until a user of the viewer 266 commands the entry to be 

17 displayed. 

1 8 Upon selection of a component with underlying links, the software directive determines 

1 9 the identifying index values associated with the selected component, searches the table for the 

20 index values of the selection made, looks up the corresponding hnked location, accesses the 

2 1 location, and displays the hnked material on the viewer 266. The linked material can be 

22 displayed on the viewer 266 in place of tiie original source material, or simultaneously with the 

23 original source material by displaying the linked material in a picture-in-picture window, via a 

24 split screen, or via a screen overlay. 

25 Figure 20 shows a portion of a page of electronic text having one or more electronic 

26 links. The display 602 may include a show links button 606 and a help button 6 12. Theshow 

27 links button 606may be used to display a link menu 971 (see Figure 13 andFigure21). That 
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1 is, the viewer 266 can be commanded, via the show links button 606, to display all components 

2 that have underlying links. The components may be displayed in a highlighted mode, in a 

3 different color, in a unique font, bold or italic typeface, underlined, outlined, or in reverse 

4 background mode, for example. To make a selection of a component to view the underlying 

5 linked material, the cursor 745 is used to identify the desired selected item. The ball 743 is 

6 used to guide the cursor 745 across the page to the desired selected item, and the selection 

7 button is used to make the selection. Altematively, cursor movement for screen navigation can 

8 be provided via devices such as a fmgerpad, mouse, or joystick. Selection can also be made 

9 by incorporating a touch-sensitive screen into the viewer 266. 

10 Figure 20 depicts the display before commanding the viewer 266 to show links. Figure 

11 21 shows the display once the request has been made to display all underlying links. Figure 

12 21 shows the link menu 97 1 , a help button 6 1 2, and a multi-function button, or pull-down 

13 menu, 614. The multi-function button 6 14 can be used as a retum button, a hide active links 

14 button, and a restore active links button, forexample. Altemately, several additional buttons 

1 5 may be provided to select these features. The multi-function button 614 may be active when 

1 6 the show links button 606 has been operated. In Figure 2 1 , the components having underlying 

17 links are "PorchScene by Cezzanne" and "Cezzanne." The component 'TorchScene by 

1 8 Cezzanne" is shown with one link and the component "Cezzanne" is shown with three links. 

1 9 Also shown in Figure 2 1 is the link menu 97 1 that lists the links, or components, the link 

20 number and a description of the linked material. For example, the material linked to the 

21 component "PorchScene by Cezzanne" is a JPEG video file showing the painting. 

22 Once a link is selected, an on-screen retum button 614 allows the subscriber to retum 

23 from the linked material back to the originally viewed text. The show links button 606 (see 

24 Figure 20) can be displayed on the viewer 266 either at all times that an electronic book is 

25 open, any time an underlying link exists, or only when commanded to do so from the viewer' s 

26 menu system 851. 
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1 When linked material is displayed (for example, in an overiay fashion) the subscriber 

2 can command the linked material to be placed in a hidden state by operating the hide active link 

3 button 6 1 4 or by use of the menu system 85 1 . When an active link is hidden, the restore active 

4 link button 614 is displayed. Operation of the restore active link button 614 will display the 

5 material linked by the active link. 

6 Alternatively, the on screen "Help" menu 887 (see Figure 1 3) provides access to 

7 further assistance when selecting Hnks. The Help menu 887 is accessed by operation of the on- 

8 screen help button 612. Related Unk options will be available for display on the viewer 266 by 

9 selecting the Help menu 887, using the cursor 745 to do so. The "Help" function allows the 

1 0 subscriber to select which specific links to be displayed on-screen. The links when displayed 

11 maybesimplyhighUghtedportionsoftext,ortextindiffeientcolors. All links for the displayed 

1 2 page can be selected to be displayed. Alternatively, by selecting a range of content in an 

1 3 electronic book that may have multiple underlying Unks, only links associated with tiie 

14 components within the selected range will be presented on the viewer 266 for the subscriber 

15 to select the specific hnk desired. Altematively, a fixed number ofMnks may be selected to be 

1 6 displayed on the screen at a time. Altematively, only a certain type of link may be selected to 

1 7 be displayed. The types of links that are available for display may be hsted in a pop-up menu. 

1 8 The subscriber can choose from this pop-up menu which of the links to display. For example, 

19 the pop-up menu could Hst links for a dictionary and links to an Internet web site. The 

20 subscriber could choose to display only the dictionary links. Figure 22 depicts the menu screen 

21 98 1 used to manage the subscriber' s filtering of links to view. 

22 Figure 23 is a logical representation of the components and links for the example first 

23 component "Cezzanne" shown in Figure 20. In Figure 23, the text block (page) 602' 

24 containing the furst component Cezzanne 980 is shown linked to components in other electronic 

25 files or documents. Afirstlink981 links Cezzanne 980 to a reference material component 982, 

26 which is an encyclopedic entry related to the artist. A second link 983 links Cezzanne 980 to 

27 a dictionary entry 984 that includes apronunciation key for the artist's name. A third link 985 
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1 links Cezzanne 980 to an audio clip 986, which provides an audio file describing the artist's life. 

2 If the subscriber chooses the audio clip 986, the audio file will immediately begin playing and 

3 will be broadcast on a speaker in the viewer 266, or the attached television or the attached 

4 personal computer, as applicable. The subscriber can stop the playback by operating the hide 

5 active link button 614. 

6 The links described above may also function in two directions. A biography of 

7 Cezzanne could include a link to the JPEG file showing PorchScene . The JPEG file 

8 PorchScene could be one of several still videos of the artist's work. This JPEG file could be 

9 linked to an appropriate section of the Cezzanne biography. Then, if the subscriber were 

1 0 viewing the JPEG file for PorchScene . the subscriber could display the link to the biography 

1 1 and, upon activating the link, have displayed that portion of the biography that discusses 

12 PorchScene. In this example, the same link is used to display either the JPEG file or text fi-om 

1 3 tile biography. The same hidden table can be used witii the two electronic files (i.e. , the 

1 4 biography and the JPEG file). Alternately, each electronic file may have its own hidden table. 

15 In this alternative, the same link may be referenced in each of the hidden tables. 

16 In the discussion above, each first component is linked to one or more other 

17 components. However, tiie other components (i.e., the second and third components, for 

1 8 example) may also be linked togetiier. hi addition, other components linked to one first 

19 component may be crossed-linked to other components that are linked to a second first 

20 component (identifying secondary or tertiary cross-links). For example, the JPEG file of 

2 1 PorchScene referred to in Figure 2 1 may be cross-linked to the audio file-biography shown 

22 inFigure21. Displaymg the JPEG file will result in a cross-link being identified tiiat links the 

23 JPEG file to the audio file. 

24 Thecross-linkmaybeindicatedbyhighlighting, underlining, outiining,usingaboldor 

25 an italics typeface, using a different font, and using different color text. If the cross-hnked 

26 material is selected, that material will then be displayed. In the example described above, the 

27 JPEG video file that shows the painting PorchScene can be cross-linked with Cezzanne and 
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1 its identity would be displayed on the viewer 266. The video file could be displayed foil screen 

2 or in a picture-in-picture format. The video file could also be displayed on the attached 

3 television 259 or the personal computer 261 (seeFigure2). Finally, the video file could be 

4 printed by sending the video data and a print command to a printer 262 attached to the home 

5 system 258 or to the personal computer 261 . 

6 In the table previously shown, all the links for a number of electronic books were 

7 provided in one hidden table. In an alternate arrangement, many hidden tables may be 

8 provided. For example, each electronic book may be provided with a hidden table. In this 

9 alternative, the many hidden tables could form a relational database of Unked material. As an 

1 0 example, several different electronic medical text books could each be provided with its own 

1 1 hidden table. An electronic general medical encyclopedia could also be provided with a hidden 

1 2 table. Terms that are Hsted in one of the several medical electronic text books could then be 

1 3 linked, in a relational fashion to the electronic medical encyclopedia. The several electronic 

1 4 medical text books could also be relationally linked to each other, to on-line databases and to 

1 5 other electronic files. For example, an electronic medical text book could be electronically 

1 6 linked to electronic books, databases and other electronic files maintained at a medical school 's 

17 library. 

1 8 The hidden table (either for many electronic books, or individually for each electronic 

1 9 book) may be provided by the central provider or distributor as an additional feature that is 

20 paid for separately from purchasing an electronic book. The distributor may offer many 

2 1 different levels of service, such as only linking material (components) stored on a viewer, only 

22 linking material within a particular electronic book, or Unking one or more electronic books to 

23 electronic files outside the home system 258 (e.g., linking an electronic book to a database 

24 maintained by the distributor at an Internet web site). 

25 Downloading the most current links table for an electronic book fi^om the operations 

26 center 250 or the Internet web site 279 refreshes the hidden links table, that is, any new links 

27 that have been generated by the operations center 250, for example, are added to the hidden 
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1 links table. The current links table may be downloaded in conjunction with downloading a new 

2 electronic book. Alternately, the current links table may be provided periodically by the 

3 operations center 250. 

4 As noted above, links within electronic books maybe self-contained in nature, where 

5 all the material to be linked to is resident within the same electronic book file. Additionally, 

6 hnks may also be provided between material residing on the viewer 266. Also, links may be 

7 provided to material that currently resides on the home system 258, if separate from the viewer 

8 266, Finally, links may be provided to material that must be accessed through a 

9 communications network. For example, material that is ordered or downloaded from the 

1 0 operations center 250 or the Litemet web site 279 may be linked to the electronic book. On 

1 1 screen menus can also be supported on the viewer 266 in the form of electronic book files, 

1 2 serving as a book or product catalog or service catalog that allows the subscriber to link to the 

13 operations center 250 or the Internet web site 279 to order additional electronic books or 

14 products and services at any time by selecting the desired component from the text. 

15 If the content that is to be linked to is currently not available on the electronic book 

1 6 viewer 266, the viewer 266 may prompt the subscriber to decide whether to: 1 ) retrieve the 

1 7 corresponding material immediately from the home system 258, the operations center 250, or 

1 8 the Memet web site 279; 2) wait until the next communication opportunity with the home 

1 9 system 258 or operations center 250 to retrieve the material; 3) commence direct outside 

20 communications with another communications system (e.g,, a telephone in aPSTN); or 4) stop. 

21 In one embodiment, the first components on the viewer 266 are a Table of Contents 

22 and List of Figures for a book. Making a selection from the Table of Contents and List of 

23 Figures automatically links to and displays the selected page within the electronic book file. In 

24 another embodiment, the fu-st components on the viewer 266 may be an Index of an electronic 

25 book. Selecting the desired topic and associated page will cause that page to be displayed on 

26 the viewer 266. In yet another embodiment, the first component is a footnote or endnote. 

27 When the footnote is selected, the viewer 266 provides a display of material that further 
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1 addresses the reference. In another embodiment, the first component is a word or phrase that 

2 has a further definition or clarification associated with it. By selecting the first component, the 

3 corresponding dictionary definition, foreign translation, or glossary entry will be displayed on 

4 the viewer 266. The dictionary definition or foreign translation may also be provided via an 

5 audio file. In this embodiment, electronic books can be bundled with other standard reference 

6 material or altematively, stand-alone reference material like dictionaries or encyclopedias may 

7 be accessed from within multiple electronic book files. 

8 In another embodiment, the first component is a reference to another electronic book 

9 altogether. By selecting the first component, the selected book is displayed on the viewer 266. 

1 0 In another embodiment, on-screen menu buttons will be displayed on the viewer screen 602 

1 1 that allow for a quick Unk to the Table of Contents, Index, glossary, and other key electronic 

1 2 book sections by simply selecting the item on the viewer screen 602 with the cursor 745. In 

1 3 yet another embodiment, the selected fu:st component links the subscriber to a book review or 

14 series of book reviews, providing additional information to assist in the decision of selecting a 

1 5 new electronic book. In another embodiment, the selected first component is a book titie, 

1 6 chapter title, or text in the body of a book that is linked to an audio file that serves as an audio 

1 7 narration of the selection that is played on the viewer 266. In yet another embodiment, the 

1 8 selected first component hnks to a video file (JPEG or MPEG) that can be displayed on the 

1 9 viewer screen 602. Another embodiment is a first component tiiat Unks to textual annotations 

20 and notes that have been created by the subscriber. 

2 1 One embodiment includes first or subsequent components that are electronic book tities 

22 that, when selected, links the subscriber to the operations center 250 or the Internet web site 

23 279 to allow for the ordering of the selected book. In a similar embodiment, the selected 

24 component consists of a product that, when selected, link the subscriber to the operations 

25 center 250 site or an Internet web site to allow for the ordering of the selected product. Lastiy, 

26 in another embodiment, the selected component is a topic on which there is a link to an 

27 Internet-based discussion group that addresses the topic in more detail. 
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1 Another embodiment is a link to a world watch live server/site. When a link is 

2 provided to link a word or phrase to a foreign language dictionary, the viewer 266 may display 

3 a foreign language selection feature. The subscriber may then indicate which language to use 

4 when activating the link. For example, an EngUsh word or phrase in the electronic book may 

5 be linked to a French, Spanish or German dictionary. The subscriber may specify which of 

6 these foreign language dictionaries to link to. 

7 IX. Connection To World Watch Live 

8 The viewer 266 may also be used to display live or recorded video. The video may 

9 be distributed using a telecommunications system, including delivery using the Internet. Access 

10 to video maybe provided using links within an electronic book or on an electronic book menu 

1 1 display. 

1 2 A. Obtaining Video From Remote Sites, Conmiunicating the Video to a Web 

13 Site, and Streaming the Video To Users. 

1 4 Figure 24 shows a system that provides video, audio and text data from remote video 

15 sources such as videocassettes and television programs. Figure 24 shows remote sites 2102, 

16 remote cameras 2104, videocassette 2106, compression devices 2108, 21 14, digital storage 

17 device 2 1 10 and web site 2 1 12, As shown in Figure 24, a video camera 2104 is used to film 

1 8 activity at remote site 2 1 02. As discussed below, numerous video cameras at a single remote 

1 9 site may be used to obtain different views, text, and audio (for example, stereophonic) of the 

20 remote site from different angles and orientations. Also, numerous remote sites, each with its 

21 own video camera, may used as shown at 2102', 2102" and 2104' and 2104". The video 

22 cameras film events at the remote sites, and record the events on videocassette 2 106 or other 

23 suitable media. 

24 The recorded information is then transported to a web site 21 12, or to a site in 

25 communication with the web site 2 1 1 2. As shown in Figure 24, the recorded information from 

26 video tape 2 1 06 is then compressed in compression unit 2 1 08 and stored in digital storage 

27 media 2110. Many compression algorithms may be used, such as MPEG- 1 , MPEG-2 and 
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1 



Wavelet. Compression systems currently available from The Duck Corp, Xing Technology 

2 Corp., Indeo, Digital Video Arts, Ltd., VDOnet Corp. andlntel Corp., may be used with the 

3 system. The digital storage media may be any known storage device, such as a hard disk, CD 

4 ROM, digital video disc (DVD), digital tape, video file server or other media. 

5 The stored and compressed data may then be provided on a number of streamed 

6 audio-video outputs 2116 from the web site 21 12. This enables many users to access the 

7 stored video, text, and audio, and allows for one user to receive numerous audio-video signals, 

8 i.e. split the display into numerous "camera" feeds. 

9 In addition to providing streamed audio and video from videocassette, the web site 

10 2112mayprovideaudio,text, and video from television channels. The television signals are 

1 1 received by a conventional television receiver (not shown in Figure 24), and digitally 

1 2 compressed by the compression unit 2 114 and fed through the web site 2 1 1 2 to the streamed 

1 3 output. It is not normally necessary to store the television programs in a digital storage unit 

14 (such as the storage unit 2 1 10), since the audio, text, and video is constantly incoming and 

1 5 changing. However, certain segments of broadcast television may be stored in a storage 

1 6 device, such as the digital storage unit 2110, for recall by a user. 

^ "7 Figare 25 shows another system for supplying video, audio and text data, where similar 

18 referencenumeralsindicateitenisthatcorrespondtotheitenisshownmFigure24. Thesystem 

1 9 of Figure 25 uses remote cameras and a communication network to provide remote video to 

20 the web site. Figure 25 shows remote sites 2 102, video cameras 2104, compression unit 

21 2118, data communication network 2120, web site 2130, digital storage unit 2132, and 

22 streamed video 2116. 

23 As shown in Figure 25, the remote sites 2102 are filmed by cameras 2104 (as in Figure 

24 24). However, in this system, the outputs of the cameras 2104 pass through a compression 

25 unit 2118. The compressed audio, text, and video is communicated over the data 

26 communication network 2 1 20 to the web site 2 1 30. The data communication network 2 1 20 

27 may be any network currently known to one of ordinary skill in the art, such as land-leased 
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1 lines, satellite, fiber optic cable, microwave link or any other suitable network. These and other 

2 delivery systems are described in more detail in Section VU. 

3 Other suitable networks maybe cellular networks orpaging networks. In apaging 

4 network, the cameras 2 104 may be connected to a paging device and/or digital storage media 

5 or paging transmitter for communication of the video (including text and still images - as used 

6 hereinafter, video refers to moving and still images, and to text data) to the web site 2 1 30. The 

7 foUowingpublications.herebyincorporatedbyreference, disclose relevant systems: PCT 

8 Publication No. WO 96/07269, published March 7, 1996 by Jambhekar et al.; PCT 

9 PublicationNo. W096/21173,publishedJuly 11, 1996byHarrisetal.;PCTPublicationNo. 

10 WO 96/2 1205, pubhshed July 1 1, 1996 by Harris et al. 

1 1 The web site 2 1 30 in this example is adapted to receive information from the data 

1 2 communication network 2 1 20. The web site may pass the video from cameras 2 104 to users 

13 at streamed video outputs 2116. Alternatively, the web site may contain a decompressor to 

14 decompress the video prior to streaming it to users, or change the compression scheme of the 

1 5 video to one which is compatible with the connected user. Alternatively, the video may be 

1 6 compressed at the streamed video output and users who connect to the web site 2 1 30 may run 

1 7 decompression software. The web site 2 1 30 may store the audio and video received over data 

1 8 communication network 2 120 in digital storage unit 2 1 32 before providing it to the streamed 

1 9 outputs 2116. Alternatively, the audio and video may be directly passed to the streamed 

20 outputs 2116. 

2 1 Figure 26a shows another embodiment of the invention diat combines the embodiments 

22 of Figures 24 and 25 and adds remote camera control. Figure 26a shows remote sites 2 102, 

23 cameras 2104, computer 2134, video path 2122, 2129, control path 2124, 2126, 2128, 

24 compressors 2108, 21 14, 2118, 2136 data communication network 2120, web site 2140, 

25 digital storage means 2 1 32, and streamed video 2 1 1 6. As with Figures 24 and 25, remote 

26 sites 2 102 are filmed by the cameras 2104. As with Figure 24, the web site 2140 is able to 

27 receive data from the video tape 2 106, compress the audio and video in the compression unit 
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1 2 1 08 , and store the compressed audio and video in the digital storage unit 2110. Audio and 

2 video from television stations may also be compressed by the compression unit 21 14 and 

3 stored or passed as streamed video 21 16, as in Figure 24. 

4 Likewise, the cameras 2 104 may be connected to the compression unit 2 11 8 (as in 

5 Figure 25) and communicate compressed audio and video to the web site 2 140 via the data 

6 communication network 2 120. Thus the functions performed by the systems shown in Figures 

7 24 and 25 may be combined in a variety of manners at the single web site 2 140. 

8 Figures 26a and 26b add the additional feature of camera control to the previously 

9 described systems. As shown in Figure 26a, the computer 2 1 34 is connected to the remote 

10 cameras 2 104. The computer is able to control a mechanical or electrical device on the 

1 1 cameras2104,toalterthecameras'orientation(includingpositionand/orangle). Audioand 

1 2 video from the camera 2 1 04 passes to the computer 2 1 34. The video may be processed and 

1 3 stored in the computer. As shown m Figure 26b, the computer may be connected to multiple 

14 remote cameras 2104' and 2104" so that multiple users may each control a camera. The 

1 5 computer 2134 may either contain a compressor or be connected to an extemal compression 

16 unit 2136. The video from cameras 2104' and 2104" is compressed and provided to data 

1 7 communications network 2 1 20. This compressed video is subsequently received by the web 

18 site2140. Theremotecameras2104',2104"(Figure26b)maybecontroUedbycontrDlsignals 

19 passed from the computer 2 1 34 on the path 2 1 24. The control signals are received by the 

20 computer 2134 from the data conmiunications network 2 1 20 over the camera control path 

21 2126. The web site 2140 provides the control information to the data communications 

22 network 2 120 over path 2 1 28 . The web site 2 140 in this system is adapted to pass control 

23 signals to the cameras 2 104 and to store video images in the digital storage unit 2132. The 

24 web site 2 140 provides a number of streamed video outputs 2116. 

25 The system shown in Figure 26a allows remote users to control the angle or orientation 

26 ofcameras 2104", 2104". Users are connected to the web site 2 140 and receive the streamed 

27 video 2116 from the cameras 2104', 2104". If the users wish to move the cameras 2104', 
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1 21(M''totheright4heymayenterausercommand(suchas"panright'') at their terminal. The 

2 terminal may be an electronic book home system 258, a personal data apparatus (PDA) or a 

3 personal computer, for example. The command is received by the web site 2140, and 

4 formatted, if necessary. The command is outputted to the data communication network 2 1 20 

5 as a control signal through the camera control path 2128. The computer 21 34 receives the 

6 camera control signals from the communication network 2 1 20 over the camera control path 

7 2 1 26. The remote computer 2134 may be adapted to control multiple cameras at multiple 

8 locations 2102, or multiple cameras at the same location 2102. 

9 When using the home system 258 to connect to a world watch live web site, the user 

1 0 may enter commands usmg the viewer 266. In this embodiment, the viewer 266 may display 

1 1 command options in a "soft key" format. The user initiates a command through use of a touch 

1 2 screen device or a pointing device such as a track ball, for example. The user interface aspects 

1 3 of the viewer 266 will be described later in more detail. The home system 258 and the viewer 

14 266 maybe adapted to display data from the world watch live web site 2140 by the addition 

15 of software routines. The software routines may be contained on a computer-readable 

1 6 medium, such as a floppy disk, for example. Alternatively, the software routines may be 

1 7 downloaded to the home system 258 and the viewer 266 over the Internet from the web site 

18 2140. 

1 9 The computer 2 1 34 is connected to the remote camera 2 104 by a camera control path 

20 2124. This path allows control commands from the computer to travel to the cameras 2104', 

2 1 2104" and control the cameras 2104", 2104". The cameras 2104', 2104" may have computer- 

22 controlled swivel motors (not shown) for panning left and right, may have a computer- 

23 controlled pivot motor (not shown) for panning up and down, and may have a computer- 

24 controlled motor (not shown) for moving a zoom lens. These motors are known to the artisan 

25 and are currently available. A number of cameras may be provided at a single site to allow 

26 multiple users to have camera control at the same time. 
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1 This system of obtaining and/or storing video at a web site is extremely flexible. The 

2 system allows for perceived camera control by multiple cameras, actual camera control of one 

3 or more cameras, perceived camera control via a wide-angle lens on a single camera, and for 

4 the generation of comprehensive interactive programs. 

5 B. Perceived Camera Control With Multiple Cameras 

6 In an alternative system, shown more clearly in Figures 27-29, users are given the 

7 perception of camera control. Toachievethis,apluraUtyoffixedcameras2104,2150,2152, 

8 2153, 2154, 2156, 2158, 2160, 2162 (Figure 27) are disposed around a remote site 2102. 

9 In accordance with this embodiment, it appears to users that they are controlling the angle or 

1 0 position of a camera when in actuality they are merely being transferred to the video output of 

1 1 a different camera. Figures 27 - 29 show this concept in greater detail. 

12 As shown in Figure 27, a building 2 146 is being prepared for demolition. Disposed 

13 around the building 2146 are cameras 2104, 2150, 2152, 2153, 2154, 2156, 2158, 2160, 

14 2 1 62, connected to a computer 2135. The computer 2 1 35 is connected to a communication 

15 network2120 (not shown). The video from cameras 2104, 2150, 2152, 2153, 2154, 2156, 

16 2158, 2160, 2162 is digitized and preferably compressed prior to communication over 

17 network 2120, either by compressors connected to the cameras (not shown) or by a 

1 8 compressor connected to the computer 2 135 (not shown). The cameras may be digital 

19 cameras or analog cameras connected to an analog-to-digital converter. 

20 The cameras specifically identified around the periphery are cameras 2 1 50, 2 1 52, 

21 2153, 2154, 2156, 2158, 2160, and2162. Forreference, thebuildingcontains the letter "A" 

22 and the letter "B" on two sides as shown at 2144 and 2148 in Figures 27 and 28a - 28d. A 

23 number of additional cameras 2 1 04 are disposed about the periphery of the building in a 

24 circular pattem. The pattern and number of cameras are not critical, but will control how the 

25 user perceives movement of the "camera". 

26 Referring to Figure 27, a video camera 2 1 50 faces side A, a video camera 2 1 52 is 

27 between sides A and B, a video camera 2153 faces side B and a video camera 2154 is 



-66- 



Docket 5281 /PTO Filings/Spec wpd 

1 between side B and the side opposite side A. The video cameras 2 1 56, 2 1 58 , 2 1 60 and 2 1 62 

2 are disposed closer to the building, as shown. All the video cameras contain audio pickups 

3 (preferably stereo). Additionally, all the video cameras are connected to a computer 2135 

4 which outputs compressed audiovisual signals to the communication network 2 1 20 and 

5 consequently to the web site. The system shown in Figure 27 may be implemented by the 

6 systems shown in either Figure 25 or Figures 26a and 26b. Any number of users in 

7 communication with the web site 2 1 30, 2 140 may receive the audio and video from these 

8 cameras. 

9 Figure 28a shows a typical screen view 2 1 50 of the video presented on the viewer 266 

10 to remote users who are connected to the web site of the present invention. As shown, the 

1 1 user is observing live video from camera 2 1 50, which provides a view of the building on side 

12 A. A"toolbar"ofcommands2151 is presented to the user, including a pan left command"^", 

13 a pan right command a pan up command "t" and a pan down command "i". An 

14 "autopan" command is used in conjunction with another command (such as pan right). The 

1 5 "autopan" command is used to automatically move the picture position in the direction 

1 6 previously entered. For example, if "autopan" is entered after "pan right," then the picture will 

1 7 keep panning right until another key is pressed or a default key (such as the ESCape key) is 

1 8 pressed. The speed of the "autopan" fimction is controlled by the "speed" command, which 

19 is used in conjunction with the "+" and "-" commands. Additionally, the "+" and "-" 

20 commands, when used alone, control a"zoom-in" and "zoom-out" fiinction, respectively. The 

2 1 "toolbar" commands are selected via a user input device, which may be a touch sensitive 

22 screen, a keyboard, mouse, trackball, and a remote confrol. 

23 When any user wishes to switch from the view of the camera 2 1 50 (Figure 28a) and 

24 pan to the right, the user initiates a pan right command "->", which is transmitted to the web site 

25 2130, 2140 (Figures 25 and 26a and 26b). The web site receives the command, and in 

26 response, causes the video from the camera positioned to the right of the camera 2 1 50, in this 

27 case the video camera 2 1 52 (Figure 27) to be transmitted to the user. The user then observes 
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1 the picture appearing in Figure 28b, which appears to be a view to the right from the previous 

2 position (camera 2 1 50). If the user continues to pan right, the user is presented with the Figure 

3 28cview,ieceivedfromthecamera2153. The user may continue to pan right all away around 

4 the building in this manner. 

5 Additionally the user has special functions available, such as "autopan" and "zoom." 

6 For example, "autopan" in conjunction with "pan right" would cause the view of the building 

7 to rotate, at a speed dictated by the "speed" function and the "+" and "-" keys. Using the 

8 "+" and "-" keys alone causes the view to change to a closer camera ("+") or a camera fiuther 

9 away ("-"). As shown in Figure 27, the cameras 2156, 2158, 2160 and 2162 are disposed 
Q 10 closer to the building than cameras 2 150, 2 152, 2 153 and 2 154. A "magnified" image, 

11 obtamed from the camera 2 156, is shown in Figure 28d. Ifno cameras are disposed closer 

j 1 2 or further away, digital image processing may be used to digitally increase or reduce the size 

\M 13 of the image. The software which controls these functions m^ be disposed either at die web 

,t; 14 server or on the user's computer. 

Q 15 Thus, users may obtain different views of the building 2 146 as if they were remotely 

;| 16 controlling the positioning of a single remote camera. The users may observe the demolition 

^17 of the building from many exciting perspectives. This "perceived" camera control is 

18 advantageousbecauseitallowsanynumberofusersto"contror'acamera. Asinglecamera 

1 9 which is remotely controllable is only controllable by a single user. Thus, the present invention 

20 is suitable for large audiences. The realism of this perceived control is directly dependent upon 

2 1 the number of cameras and their distances from the viewed object. 

22 Therefore, when thebuilding2146 is demolished, any number of users may pan around 

23 the building in real time as if they were actually present at the site. When the building is 

24 demolished, the video cameras pick up, preferably in stereo, the sounds of the demolition. 

25 Users who have loudspeakers connected to their computer may experience the demolition 

26 almost as if they were present. 
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1 Figure 29 shows a deployment of a number of cameras 2 104 which are arranged in a 

2 linear fashion around a point of interest, each camera connected to computer 2 1 35 as in Figure 

3 27 . As with Figures 27 and 28a and 28b, this embodiment uses "perceived" camera control 

4 which may be achieved by the systems shown in Figurcs 25, 26a and 26b. In this example, the 

5 remote location and point of interest is a parade, such as a New Year's Day Parade. With the 

6 camera deployment shown, a user may traverse the length of the parade without actually being 

7 present. Users may view whichever part of the parade they are interested in, for as long as 

8 they desire, without worry that they have missed an interesting band or float. In this example, 

9 the camera deployment merely follows the parade route. Parents who have children in a band 

10 or float may search for the child and follow the child throughout the parade route, rather than 

1 1 having to monitor every moment of the parade on television in the hopes that the child will pass 

12 the reviewing camera when the parents are watching. The parents merely "move" from 

13 different cameras along the parade route as their children progress in the parade. 

14 C. Actual Camera Control of Single/Multiple Cameras 

1 5 Figures 30a and 30b show another system, where a number of cameras 2 1 60, 2 1 62, 

16 2164, 2166, are provided. These cameras are in direct communication with and are controlled 

17 by computer 2 170. Although it is possible to form a ring of cameras to perform "perceived" 

1 8 camera control (as in Figures 27 - 29) , the embodiment shown uses four cameras 2 1 60, 2 1 62, 

19 2164, 2166 which contain motors 2105 (Figure 30b) for controlling the camera's positioning. 

20 The motors are controlled by computer 2 170. Either a single computer 2 170 or a number of 

2 1 computers 2 170 may be used. The remote location and point of interest shown in Figures 30a 

22 and 30b are, for example, a watering hole or desert oasis. Users who access the web site 

23 2 140 are able to observe live video of wildlife behavior at the watering hole. The cameras 

24 2160, 2162, 2164, 2166 are disposed at an island in the middle of the watering hole. The 

25 toolbar 215 1 of Figure 28a is also used in this embodiment and enables users to choose 

26 camera control commands to spin the cameras around or perform other camera functions, such 
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1 as zoom. Users are therefore able to receive different views and angles, and observe the entire 

2 watering hole. 

3 Figure 30b shows the control and video paths of the Figure 30a system combined with 

4 system shown in Figures 26a and 26b. The video from cameras 2 1 60, 2 1 62, 2 1 64, 2 1 66 is 

5 communicated to computer 2 170, in compressed or uncompressed form on path 2 1 22. The 

6 computer 2 170 communicates the video to communications network 2 120 for reception by the 

7 web site 2 140 (Figures 26a, 26b). Preferably the video is digitized and compressed by either 

8 the cameras 2160, 2162, 2164, 2166, the computer 2170, or an external analog-to-digital 

9 converter (not shown) and compressor 2136 (Figures 26a, 26b) prior to transfer to the 

1 0 communications network 2 1 20. 

1 1 Camera control commands are received by the computer 2 170 on control line 2 126, 

12 as shown in Figures 26a, 26b and 30b. The conmiands are formatted, if necessary, by 

1 3 computer 2170 and transferred to control units 2105 attached to cameras 2 160, 2162, 2164, 

14 2 166. The control units 2105 are connected to spin, zoom, or otherwise control the cameras 

15 as directed by the user. 

1 6 Communications links 2 124 and 2122 maybe wired, wireless, digital or analog, and 

1 7 computer 2 1 70 may be located nearby or remote from the site 2 1 02. 

1 8 The system of Figures 30a and 30b are unlike the systems shown in Figures 27 - 29, 

1 9 because each user is assigned a remote camera in the Figure 50a, 50b embodiment. Since 

20 each user must be assigned their own controllable camera, users will have to contend for 

2 1 available cameras. The number of controllable cameras may range from a single camera to any 

22 number, and is preferably statistically determined to correlate to the average number of users 

23 who access the web server 2 140 at any given time or at peak times. The number of cameras 

24 may be reduced by using known systems which utilize queuing, reservations, and time limits. 

25 D. Perceived Camera Control Using A Single Camera And A Wide- Angle Lens 

26 Figures 3 1 a and 3 lb show another embodiment, using only a single camera, where an 

27 unlimited number of users may view any portion of the remote site 2 102. This embodiment 
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1 uses a spherical lens 2 1 82 in optical communication with the camera 2180. The remote site 

2 2 1 02 shown in Figure 3 1 a is a remote watering hole or oasis as in Figures 30a and 30b. 

3 As shown in Figure 3 1 a, a camera 2 1 80 has a spherical (or other wide angle) lens 

4 2182, which provides a 180°, spherical or other wide-angle view. This view, which is 

5 communicated to a computer 2184, contains distortion. The computer 2 1 84 communicates 

6 and compresses the distorted video back to the web site 2 1 30 or 2 1 40 which stores and may 

7 process the image. Rather than using the computer 2 184, a simple transmitter may be used to 

8 convey the entire spherical video to the web site 2 1 30, 2 140 (Figures 25 and 26a and 26b). 

9 By using appropriate image processing software, the web site removes the barrel distortion and 

10 stores data relating to the entire spherical view. Users maythen access different portions of 

11 the 180° sphere. In this embodiment, the toolbar 2 15 1 ofFigure 28a is also used. Byusing 

1 2 the toolbar 2151, users may move across the spherical view and obtain the "perception" of 

1 3 camera control. This embodiment is advantageous in that it can provide the perception of 

14 camera control to any number of users simultaneously using only one remote camera. 

15 Figure 3 lb shows alternative embodiments ofthe system shown in Figure 3 la. As 

1 6 shown in Figure 3 1 b, the spherical (or other wide angle) lens 2 1 82 is used with video camera 

1 7 1 80', which conveys video information to computer 2 1 84. Computer 2 1 84 communicates the 

1 8 video over communications network 2120 to the web site 2130. The web site 2130 may store 

1 9 or process the received video, and make the video available to users at user terminals 2302, 

20 2304, 2306, 2308, 23 10 by communicating the video over communication network 2 125. 

21 Communication network 2125 is explained in more depth below with respect to Figure 33. 

22 Because wide angle lenses generate distortion, processing is conducted on the distorted 

23 image to remove the distortion from a segment of the image. This processing may be 

24 performed at the computer 2184, or the web site 2 1 30. Alternatively, the processing maybe 

25 performed at the user terminals 2302, 2304, 2306, 2308, 23 10. 

26 Thus, the web site 2 1 30 has available wide angle video for sending to users. Users 

27 display and view only a segment of the wide angle video at a time. Then, by using toolbar 2151 
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1 (Figure 28a), the user may select adjacent segments of the video for view. When a user selects 

2 an adjacent segment of the video for display, the adj acent segment is processed to remove 

3 distortion and then displayed. Displaying the adjacent segment gives the appearance that the 

4 camera v^as physically "moved" to the adjacent side of the original segment. 

5 One system for electronically removing the distortion from a segment of an image 

6 obtained from a fish-eye lens is disclosed in U.S. Patent No. 5, 1 85,667, issued February 9, 

7 1 993 to Zimmerman, incorporated herein by reference. Zimmerman' s apparatus uses the 

8 following hardware for processing a captured and digitized image: a microcomputer connected 

9 to a remote control, computer control, X-Map and Y-Map; an input image buffer connected 

10 to the X-Map and Y-Map with an output connected to an image filter and an output image 

1 1 buffer. This hardware, for example, or any other suitable hardware, may be placed at the 

1 2 computer 2 1 84, or the web site 2 1 30, but is preferably located at the user terminals 2302, 

13 2304,2306,2308,2310. 

14 Alternatively, the specialized hardware is removed and the hardware functionality is 

1 5 implemented in software at the computer 2 1 84 or web site 2 1 30, but preferably the software 

1 6 is loaded into the user terminal 2302, 2304, 2306, 2308, 23 10. Thus, in accordance with the 

17 present invention a spherical (or other wide-angle) image is suppliedto the user's terminal, 

1 8 which executes appropriate software (which may be a "plug-in" for a browser application 

1 9 program) for displaying a segment of the image (or video) without distortion. Additionally, the 

20 distorted spherical image (or video) may be saved to a storage medium, either at the user' s 

21 terminal or at the web site, for future loading and viewing. 

22 Figure 3 lb also shows how to remove the lens distortion without special processing. 

23 As shown in Figure 31b, a spherical (or other wide angle) lens 2182 is in optical 

24 conmiunication with a video camera 2 1 80'. However, a nonlinear imaging sensor 2 1 86 is 

25 placed between the spherical lens 2 1 82 and the video camera 2180'. The imaging sensor is 

26 designed to provide a distorted output which cancels out the distortion of the spherical lens 

27 2182, and thus an undistorted wide-angle image is provided to video camera 2180'. 
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1 Alternatively, imaging sensor 2186 may itself provide a digital output, making it unnecessary 

2 to use a camera 2 1 80'. In this case, the imaging sensor 2186 would be directly connected to 

3 computer 2184. 

4 Examples of imaging sensors 2186 are disclosed in U.S. Patent No. 5,489,940, issued 

5 on February 6, 1 996 to Richardson et al., and in PCT publication WO 96/12862, published 

6 June 13, 1996 to Richardson et al, each incorporated herein by reference. Other suitable 

7 imaging sensors may be used with the present invention. 

8 The image obtained by the imaging sensor 2186 may be undistorted and not require 

9 ftirther processing. A segment of the image may then be selected for display by simply passing 

1 0 the image data to a display device. If the imaging sensor is imperfect, further processing may 

1 1 occur to correct for defects in the sensor. Additionally, further processing for "zoom'' and 

1 2 "unzoom" functions may occur. This further processing may take place at the web site 2 1 30 

13 or at the user's terminal 2302, 2304, 2306, 2308, 23 10. 

14 The embodiments of Figures 48a through 5 lb may be used in conjunction with either 

1 5 live audio and video or prerecorded video data (with audio) (shown in Figures 24 - 26b). For 

1 6 example, if nothing interesting is happening at the watering hole, a connected user may access 

1 7 a stored audio and video clip of a lion attack which occurred the day before. If "perceived" 

1 8 camera control is utilized, the stored audio and video preferably includes all camera angles (or 

19 a wide-angle view), such that the ability to pan and zoom is preserved, 

20 E. Web Site Configuration 

2 1 Figures 52a and 52b show a more detailed view of the web site, listed as web site 

22 2140 (Figure 28a), but which may also correspond to web sites 2112 (Figure 24) and 2130 

23 (Figure 25). The web site 2140 is connected to a data communication network 2120, the 

24 Internet 2242, and direct connections 2244. The web site contains transmission equipment 

25 22 10, receive equipment 2220, 2220', two compression units 2 1 08, 2 1 1 4, a web server 2200, 

26 a router 2230, and communication equipment 2240. The web server 2200 itself contains a 

27 digital matrix switch 2250, a plurality of digital video servers 2252, 2252', 2252", 2252'", a 
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1 firewall access control unit 2254, a database server 2256, an audio and video storage unit 

2 2258, a data storage unit 2260, an administrative unit 2262, a digital matrix switch 2264, a 

3 camera control unit 2268 and a digital video matrix switch 2270. 

4 The web site 2140 is connected to the data conmiunication network 2120 by 

5 transmission equipment 22 1 0 and receive equipment 2220. As shown, multiple receivers 

6 2220, 2220' may be used. Also, as shown, the receivers may have more than one video 

7 output. Audio and video signals may also be input to the web server 2200 by videocassette 

8 (or other suitable recorded media) or simply by feeding in television programming. As with 

9 Figures 24 and 26a, these signals are preferably compressed by compression units 2108, 

10 21 14. On the opposite side, the web server 2200 is connected to remote users by a router 

1 1 2230 and communication equipment 2240, which in turn are connected to the Intemet 2242 

1 2 or directly connected 2244 to users. The communications equipment 2240 outputs the video 

1 3 streams 2116 through a number of input/output ports. 

14 As previously stated, the web server 2200 contains a digital matrix switch 2250, a 

1 5 plurality of digital video servers 2252, 2252', 2252", 2252,"' a firewall access control unit 2254, 

1 6 a database server 2256, an audio and video storage unit 2258, a data storage unit 2260, an 

1 7 administrative unit 2262, a digital matrix switch 2264, a camera control unit 2268 and a video 

1 8 matrix switch 2270. 

1 9 The digital matrix switch 2250 receives all incoming compressed video signals from the 

20 receivers 2220, 2220' and the compressor units 2108, 21 14. The matrix switch 2250 also 

21 receives compressed video data from database server 2256, Under control of the 

22 administrative unit 2262, the digital matrix switch 2250 outputs the input compressed video 

23 signals to digital video servers 2252, 2252', 2252", 2252"'. In this manner, any input signal can 

24 be transferred to any video server as dkected by the admin unit. Also, stored programming 

25 from the database server 2256 is routed to the digital matrix switch 2250 to be switched as if 

26 it were incoming live video. The outputs of the digital matrix switch 2250 also connect to the 
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1 database server 2256, so that anything at the inputs, such as incoming live audio and video, can 

2 be stored in the database server 2256. 

3 The compressed input video is passed into various digital video servers 2252, 2252', 

4 2252", 2252'" for formatting. Users who connect to web server 2200 preferably run their own 

5 decompression software so that the no decompression need occur at the web server 2200. 

6 As an alternative, the digital video servers may decompress the input video. 

7 The audio and video from the video servers 2252 are passed through a second digital 

8 (video) matrix switch 2270. Since switching has already occurred at the digital matrix switch 

9 2250, the second video matrix switch 2270 is not required, but is desired for maximum 

10 flexibiUty. It is also optimal where the number of users exceeds the number of video inputs, as 

1 1 one input may be channeled to numerous connected users. 

1 2 The matrix switch 2270 may contain a processor that joins different frames of video 

1 3 and audio such that each output contains frames for multiple video pictures (including audio). 

14 This enables users to receive split screen images of video and select an audio track for 

15 playback (see Figure 37, discussed below). The split-screen images maybe formed by using 

1 6 known methods, which may differ depending on the type of compression used. For example, 

1 7 digital images may be decompressed, combined with other decompressed images, and then re- 

1 8 compressed; or the images may be decompressed and converted to analog, combined, and 

19 then converted to digital and compressed for transmission. 

20 The signals switched by the video matrix switch 2270 are preferably digital. This is 

2 1 because the communicated video streams 2 1 1 6 are preferably digital. It is preferred to process 

22 all the signals in the web server in the digital domain to improve simplicity and maintain 

23 maximum flexibility. 

24 The various streams of video output from the video matrix switch 2270 are passed to 

25 the firewall access control unit 2254 for output to the router 2230 and the communication 

26 equipment 2240. 
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1 Using this system, any user may receive any signal present at any input, including storcd 

2 signals within audio and video database 2258 or data storage unit 2260. Additionally, any 

3 compressed digital signal present at the input to digital matrix switch 2250 may be stored in the 

4 audio and video storage unit 2258 or data storage unit 2260. This is advantageous in the 

5 perceived camera control embodiment (Figures 27 - 3 lb) where the web server 2200 must 

6 output a different video picture to the user upon user request. When the user request is 

7 received by the web server 2200, the administrative unit 2262 directs the matrix switches 2250 

8 and 2270 to output the correct video stream to the user. If the user is requesting stored video, 

9 the administrative unit directs the database server 2256 to provide the video to digital matrix 

10 switch 2250. ff graphics or textual data are required, the administrative unit 2262 directs the 

1 1 database server 2256 to output the text or graphics to digital matrix switch 2264. 

1 2 Although shown as one functional box, the database server 2256 may be implemented 

1 3 by using several servers and/or multiport servers . The audio and video storage unit 225 8 and 

14 data storage unit 2260 may be implemented by using many storage media of different types, 

1 5 such as optical storage devices (i.e. CD-ROM), magnetic disks, magnetic tape, or memory 

1 6 circuits (i.e. RAM/ROM). The number of units depends on the amount of stored data, the 

17 number of users, and the desired output speed. The database server 2256 may be one or 

1 8 multiple units. The audio and video storage unit 2258 stores (preferably compressed) audio 

1 9 and video presentations, including all relevant camera angles. The video servers 2252 may also 

20 be implemented as one or more servers and/or multiport servers. 

2 1 The data storage unit 2260 is used to store information relating to audiovisual displays. 

22 This information relates to the menu stmcture and screen displays communicated to connected 

23 users. The stored information may also relate to specifically to the audio and video which is 

24 currently being displayed and heard. For example, in the demolition embodiment of Figure 

25 28a, a user may click on a "more info" icon, to obtain information on demolition. Such 

26 information, which could include statistics on dynamite, for example, would be stored as text 

27 or graphics in data storage unit 2260. The "more info" command would be transmitted to the 
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1 communications equipment 2240, pass through the router 2230, and the &ewall access control 

2 2254 to administrative unit 2262. The administrative unit 2262 then directs the database server 

3 2256 to recall the relevant information, such as statistics on dynamite, from data storage device 

4 2260 and pass the information to digital matrix switch 2264. The recalled information is then 

5 passed to the firewall access control unit 2254, the router 2230, and the communication 

6 equipment 2240 for transmission to the proper subscriber. The datamay be combined with 

7 audio and video in the firewall access control unit 2254, or be a separate transmission. 

8 In the perceived camera control embodiment, the conmiunication equipment 2240 

9 forwards the user' s command (such as "pan right") to the router 2230, which detects the 

10 command and forwards it to the firewall access control unit 2254, which passes it to the 

1 1 administrative unit 2262. The administrative unit 2262 controls the video being fed to each 

1 2 connected user. The administrative unit 2262 also responds to user commands by instructing 

1 3 either the matrix switch 2250 or the matrix switch 2270 to pass a different audiovisual signal 

1 4 from another source (i.e. camera, for example, the camera to the right of the present camera) 

15 to the connected user. If the user is receiving a stored image from database 2258, the 

16 administrative unit instructs the database server 2256 to recall the appropriate video 

17 signal. 

18 In the actual camera control embodiment (shown in Figures 26a and 30a), commands 

1 9 from the user (such as "pan right") are received by the communication equipment 2240 and 

20 forwarded to the router 2230. The commands enter the web server 2200 via the firewall 

2 1 access control unit 2254, and are passed to the administrative unit 2262. The commands may 

22 be stored in the administrative unit 2262 or passed to the database server 2256. Either way, 

23 the conmiands pass through the camera control unit 2268 which formats the commands as 

24 necessary for remote camera control. The formatted commands are passed to the transmission 

25 unit 22 1 0. The transmission unit 22 1 0 provides the commands to data communication network 

26 2120 for reception at remote cameras and CPU 134 (Figure 26a). 
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1 Li the system with the spherical (or other wide angle) lens(sho wn in Figures 3 1 a and 

2 31b), where the remote camera uses a spherical lens 2 1 82, the administrative unit 2262 

3 determines which segment or quadrant of the audiovisual image is to be supplied to the user in 

4 response to the user' s command. The spherical image may be stored in database 2258 prior 

5 to being output to digital matrix switch 2250. The image is split into a number of sections, 

6 which when combined form the entire 180° sphere. By using suitable image processing 

7 software, the distortion is removed or minimized in each segment. The administrative unit 2262, 

8 in response to a user command, determines which segment of the sphere should be sent to the 

9 user. The administrative unit then directs the database server 2256 to retrieve and output the 

1 0 correct segment to the digital matrix switch 2250. By controlling the digital matrix switch 2250 

1 1 and video matrix switch 2270, the administrative unit 2262 is able to ensure that the user 

12 receives the correct segment of the spherical image. 

13 In one system the entire spherical (or other wide angle) video is communicated to the 

14 ' user, and the distortion removed by software at the user's terminal. This minimizes the 

1 5 complexity of the processing necessary at the web site 2 140, and allows the user to store the 

16 entire spherical (or other wide angle) video. 

1 7 The conmaunication equipment 2240 maybe designed to automatically determine the 

18 maximum data rate at which information can be transmitted to the connected users. The data 

19 rate depends on the type of connection the web sites has with the user, and the type of 

20 equipment the user is operating. The communications equipment may use the maximum data 

2 1 rate possible as sensed from the user' s communications. Alternatively, users may enter their 

22 data rates when prompted by a menu screen, as shown in Figure 38 and described below. The 

23 dataratesarethenstoredincommunicationsequipment2240. The communications equipment 

24 2240 may also compress the video streams prior to transmission using any known compression 

25 algorithm. Additionally, the communications equipment may remove video frames, preferably 

26 prior to compression, such that the resulting data rate is reduced to be compatible with the user. 
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1 Figure 32b is identical to Figure 32a, but contains an input interface 2225 and an ou^ut 

2 interface 2235. The input interface 2225 is used to obtain digital video from other sources, 

3 such as apaging system, cellular system, cable television system, etc. The output interface 

4 connects the web site to other communications systems such as paging systems, cellular 

5 systems, or cable television systems. In the case where the input interface connects to an 

6 analog system, it contains suitable analog to digital converters (not shown). Also, where the 

7 output interface connects to an analog system, it contains suitable digital to analog converters 

8 (not shown). 

9 For example, the input interface 2225 may obtain images or video from a paging 

1 0 system, and the output interface 2225 may be connected to a paging system to broadcast video 

1 1 or images to a selective call receiver, hi this regard, the following pubUcations are incorporated 

1 2 by reference, each of which relates video/images to selective call receivers: PCT Pubhcation 

1 3 No. WO 96/07269, pubUshed March 7, 1996 by Jambhekar et al., PCT Publication No. WO 

14 96/21 173, published July 1 1 , 1996 by Harris et al., and PCT Pubhcation No. WO 96/21205, 

15 published July 11, 1996 by Harris et al. 

16 F. Communication To Electronic Book Home Systems 

17 Figure 33 shows how the users are connected to the web site using home system 258, 

18 and shows an example of a communications network 2125 (Figure 31b) in detail. The 

1 9 connections shown in Figure 33 apply to the web sites of the previous figures, including the web 

20 site2112(Figure24),2130(Figure25)and2140(Figures26aand32a). Figure 33 shows 

2 1 a server platform 2200, the Litemet 2242, two direct connection 2244, two traditional Intemet 

22 hosts 2272, 2274, two cable Intemet hosts 2276, 2278, a satellite-based Intemet host 2280, 

23 a telephone dialup 2282, an ISDN channel 2284, a cable plant 2286, 2288, a satellite system 

24 2290andaplurality of connected userterminals. The user terminals shown in Figure 33 are 

25 home systems 258 including electronic book viewers 266. However, the user terminals can 

26 be any electronic device capable of receiving and displaying digital data including personal 

27 computers, for example. 
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1 In operation, the web sites 21 12, 2130, 2140 may communicate over the Internet 2242 

2 to a number of different systems. These systems include the traditional Internet hosts 2272, 

3 2274, the cable headend Internet host 2276 and the wireless Internet host 2277. The 

4 traditional Intemet host 2272, 2274 may be connected using a telephone line 2282 or an ISDN 

5 channel 2284 to a plurality of home system 258. The cable Intemet host 2276 may be 

6 connected using a cable plant 2286 to a home system 258. 

7 Alternatively, the web site is connected using a direct connection 2244 to a cable 

8 headend Intemet host 2278 or sateUite-based Intemet host 2280. The cable headend Intemet 

9 host 2278 communicates to a cable plant 2288 and a home system 258. The sateUite-based 

1 0 Intemet host 2280 communicates using a satellite 2290 to a home system 258. These direct 

1 1 connections 2244 enable a higher data rate and use a high speed cable modem. 

1 2 The communications equipment 2240 (Figure 32a) may enable communications with 

1 3 any type of user terminal, including the home system 258, no matter what the data rate or 

1 4 system. User terminals with higher data rates will receive higher quality audio and video 

15 images. 

16 G. Screen Displays and Features 

1 7 Figures 34-39 show examples of display pages that are shown at the home system 

1 8 258. The pages and menus are stored in data storage unit 2260 (Figure 32a) as graphical 

1 9 and/or textual information. The world watch live menu may be accessed using the menu system 

20 858 shown in Figure 13. 

21 Figure 34 shows an example of a home page 2400 that contains a number of 

22 advertisements 2402, numerous web links 2404, a society link 2406, options for viewing 

23 television programming 2408, a plurality of rapid access entry options 2409 including a world 

24 watch live option 2410, and options for clubs 2412. 

25 The advertisements 2402 are useful for the page provider to generate revenue. As 

26 described previously, the system is designed such that television programming can be supplied 

27 over the Intemet. Users may view television progranuning by selecting the home page 
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1 television option 2408 . The Magazines 2404 are used to provide information concerning 

2 specific topics to the user. Users may join a society, having additional membership benefits, 

3 through the "society" selection 2406. The world watch live feature 2410, part of the rapid 

4 access entry options 2409, is selected when users wish to watch live video from remote sites. 

5 The clubs shown in the club option 2412 are selected by users who wish to obtain information 

6 related to conmion areas of interest. 

7 Figure 35 shows a society menu 2406, selected from the Figure 34 home menu page. 

8 As shown in Figure 35, there are options for world watch live 2420, is an advertisement 2402, 

9 subscription information 2424, and numerous club options 2422. This screen and all the 

1 0 functions selected in response to the displayed options may be provided on a subscription or 

1 1 temporarily free basis. 

12 Figure 36 shows one example of a world watch live menu 2440. This menu maybe 

13 used to select remote locations from which to observe live or prerecorded video. In this 

14 example, a map of the world is presented with sites that are available to select for observing 

1 5 live video. The screen indicates sites that are active 2442 or under construction 2444. This 

16 menu also contains two advertisements 2402. 

1 7 The world watch live embodiment allows connected users to visit virtually anyplace 

18 in the world to learn more about its culture, geography, or environment. Coupled with 

19 perceived or actual camera control and associated prestored video, textual and graphical 

20 information, a powerful and inexpensive learning tool is realized. This is more closely shown 

21 in Figure 37. 

22 Figure 37 shows a menu 2450 that corresponds to the Egyptian site in Figure 36. This 

23 screen concerns "Giza, Egypt", and contains live video from five cameras. As shown in the 

24 screen, there is camera one 2452, cameras two through five 2454, a ''Map" option 2456, an 

25 "About This Site" option 2458, an "About Egypt" option 2460, an "Upcoming Events" option 

26 2462 and a "Remote Control" option 2464. Camera one 2452 is the default for the main 

27 viewing camera. The user may select video image sizes and the number of images to be 
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1 displayed, limited by the equipment the user is operating. Video from cameras two through five 

2 are supplied along with that from camera one to provide altemative sites and viewpoints about 

3 the topic of the screen (i.e., Egypt). 

4 In an embodiment, the information presented in Figure 37 may be accessed by 

5 activating a link in an electronic book. The link may exist in an electronic book devoted to the 

6 history and geography of Egypt. The link may exist in an electronic encyclopedia. The link 

7 may exist as a text address (e.g., a HTML address). The link may also exist as a location or 

8 an icon in the electronic history book or the electronic encyclopedia. When the Unk is 

9 activated, video feed, either live or delayed, may be provided to the user's terminal (e.g., the 

10 viewer 266, the personal computer 261, or the television 259, shown in Figure 2). The video 

1 1 may be accompanied by audio data, text data and geographical data. 

1 2 The "Map" option 2456 brings the user back to the world map (Figure 36) to select 

1 3 additional sites. The "About This Site" option 2458 brings up text, graphics or additional video 

14 concerning the site of Giza, Egypt. For example, a professor appears and talks about the origin 

15 of the Sphinx (shown by camera 1). The embodiment shown in Figure 39 and described 

1 6 below (interactive lecture) may be combined with the "About This Site" option. Additionally, 

1 7 other video may be displayed in response to selection of "About This Site". Such video may 

18 be a documentary of the Sphinx or discussion about the technology that historians estimate was 

19 used to construct the Sphinx. 

20 The "About Egypf ' option 2460 brings up graphics, text or additional video concerning 

2 1 Egypt. For example, a map of Egypt with population densities may be shown. The option for 

22 "Upcoming Events" 2462 brings graphics, text or video concerning new events in Egypt. For 

23 example, text and newspaper articles conceming the construction of new irrigation canals is 

24 displayed. 

25 "Remote Control" option 2464 brings up a command menu (such as the "tool bar" 

26 2151 of Figures 28a - 28d) that allows the user to change camera angles or positioning in any 

27 of the cameras capable of that effect. The menu would apply to actual or perceived camera 
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1 control. For example, the user could pan around the Sphinx (camera 1, shown at 2452) to 

2 observe it from the front, each side, and back. 

3 Thus, this single screen relating to Egypt provides a wealth of information at a single 

4 Internet address (or web site). The user may, but need not,"link" to other locations on the 

5 Internet. Audiovisual presentations are displayed, which give the user insight into the people 

6 and culture of Egypt. Text, graphics, and additional stored video is available to further educate 

7 the user. Camera control (actual or perceived) gives the user the feeling of walking around 

8 different locations in Egypt. 

9 Figure 38 shows a screen 2470 that asks users about their equipment in order to 

10 determine the appropriate data rate for communications. Preferably the screen is not needed 

1 1 and the data rate is determined by communication equipment 2240 automatically. Note that 

12 an advertisement 2402 is also shown on this screen. 

1 3 Figure 39 shows an interactive lecture system. As shown in Figure 39, live video 2500 

14 of an astronomy professor' s lecture is transmitted to connected users. The users are able to 

1 5 ask the professor questions 25 1 0 and receive answers 25 1 2. The live video 2500, questions 

16 25 1 0, and answers 25 1 2 are shown to all connected users. The users may enter questions via 

17 keyboard or microphone. However, if suitable data rates are available, the user may ask a 

1 8 question via video. Thus a split screen video showing both the person asking the question and 

1 9 the lecturer may be presented to all users simultaneously. The answers are preferably given by 

20 the lecturer, who may observe the question on a remote display. Alternatively, the answers 

2 1 may be supplied by the web site as text, graphics, or prestored video . The answer may pass 

22 through a closed captioning device, be encoded, and displayed on the screen in an answer box 

23 25 1 2. Other techniques for two-way video communication are described in co-pending U.S . 

24 Patent Application Serial No. 09/39 1 ,46 1 entitled VIDEO CONFERENCING USING AN 

25 ELECTRONIC BOOK VIEWER, filed September 8, 1999, the disclosure of which is hereby 

26 incorporated by reference. 
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1 Referring to Figure 32a, questions are sent to the web site 2 140 as part of the normal 

2 user terminal communication. The web site 2 140 receives the question at the communications 

3 equipment 2240 and forwards the question through router 2230 and the firewall/ access control 

4 unit 2254 to the administrative unit 2262. The administrative unit 2262 determines whether the 

5 question can be answered by playing stored video or showing stored text or graphics. If so, 

6 the administrative unit 2262 directs the database server 2256 to recall the appropriate 

7 information. The information is then output through the matrix switches 2250, 2270 or 2264, 

8 under control of the administrative unit, as appropriate. The ability of the administrative unit to 

9 answer questions depends upon the complexity of its software. Simple, prestored answers to 

1 0 frequently asked or standard questions may be provided in a basic system. More advanced 

1 1 systems may utilize an interpreter to analyze the question before providing an answer. For 

1 2 example, frequently asked questions in the astronomy field may be "what is a star"? or "how 

13 was the galaxy formed"? In response to these questions, which may be provided on a menu 

14 or Ust, the administrative unit recalls prestored answers in either video, text, or graphics 

15 If a question cannot be answered by the administrative unit, or is sent directly to the 

1 6 remote lecturer, the question proceeds to the remote lecturer in a similar fashion as the camera 

17 control signal (Figure 26a) discussed previously. However, in the interactive lecture 

1 8 embodiment, the camera control unit 2268 (Figure 32a) is replaced with a question format unit 

19 (not shown) which reformats the question under control of the administrative unit 2262. 

20 Transmitter 22 10 then transmits a question signal to the location of the remote lecture via the 

21 datacommunication network 2120 and the communication paths 2126, 2128. The lecturer has 

22 a display which shows questions received over the data communication network. 

23 In an altemative system, the lecturer or a number of assistants may select from among 

24 many prestored answers in response to a question. In this system, the remote lecturer has a 

25 computer and monitor (not shown) which displays the questions and the available prestored 

26 answers. The lecturer or assistants then match answers with the questions. The prestored 

27 answers are preferably forwarded to the individual who asked the associated question. In 
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1 order for others to learn from the questions, the questions and answers may be provided to all 

2 connected users. 

3 Figures 40 and 41 show a system that uses acombination of live video, stored video, 

4 stored graphics, camera control and interactive questioning. The live video 2550 of camera 

5 1 shown in Figure 40 relates to a geological site, i.e., the geyser, "Old Faithful." Since the site 

6 is located on a National Park, the display screen has been customized to allow for the selection 

7 "About National Parks" 2604. When this is selected, the user' s command is communicated 

8 to the web server 21 12, 2130, 2140 for analysis by the administrative unit 2262. The 

9 Administrative unit 2262 determines that prestored video and graphics are required, and 

10 instructs the database server 2256 to output the correct information: video to the matrix switch 

1 1 2250, and graphics to the matrix switch 2264. The matrix switches, 2250, 2270, and 2264, 

1 2 under control of the administrative unit 2262, forward the video and graphics to the user 

1 3 through the communication equipment 2240. 

14 Figure41 shows the result at the user terminal. The conmiunicated prestored video 

15 2560 of a Park Ranger appears on the screen. The Park Ranger discusses the topic of 

1 6 National Parks. The discussion occurs in conjunction with a graphical display of the locations 

17 of all National Parks, shown at the screen location 2570. 

1 8 The user may select other options, such as "Map 600" to retum to the map of all 

1 9 remote sites, "About This Site" 2602 to learn more about the site currently viewed, "More 

20 About National Parks" 26 14 for even more information about National Parks, "Upcoming 

2 1 Events" 2606 for a schedule of upcoming events, "Remote Control" 2608 for remote (either 

22 actual or perceived) control of the camera (i.e. camera 1 ), "Ask Questions" 26 10 for asking 

23 questions (as in Figure 39) to an on-line Park Ranger, and "Other Topics" 26 12, for a list of 

24 other topics and/or options. 

25 H. Surveillance Systems 

26 The remote camera systems discussed above may be used in a surveillance or tracking 

27 system. For example, a researcher may place a video camera in the center of a watering hole. 
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1 preferably connected to a video recorder for storing many hours of activity at the watering hole. 

2 Multiple cameras or a wide-angle lens may be used such that virtual camera control (as 

3 described previously) may be performed on the video. Such a surveillance system has many 

4 advantages. 

5 First, the surveillance system allows for automatic scanning of the surveyed area, 

6 without the need for moving any cameras. Additionally, viewing multiple segments of the area 

7 under surveillance may be viewed at the same time in a split-screen or multi-screen image. All 

8 that needs to be done is the removal of distortion in multiple segments of the video (if using a 

9 wide-angle lens). The disclosure of U.S. Patent No. 5,359,363, issued October 25, 1994 to 

10 Kuban et al., incorporated herein by reference, discloses one example usable with the 

1 1 surveillance system. 

1 2 Second, automatic monitoring and/or tracking may be performed. Often, researchers 

1 3 and photographers wait through long periods of inactivity before a desired event occurs. For 

14 example, a photographer may wait for hours for a lion or other wildlife to approach the 

1 5 photographer' s position. The system may be used to automatically monitor a remote region 

1 6 for activity. In this case, a processor may monitor the multiple cameras or the digital wide- 

1 7 angle video for pixel changes indicating the desired event. For example, an approaching lion 

18 in an otherwise inactive desert environment will cause a moving pattem to form on a camera' s 

1 9 output or in the wide angle image. A processor may detect the pattem and alert a wildlife 

20 researcher that an event is occurring. 

2 1 Further, the processor may automatically and continually display the relevant camera 

22 output, or the segment of the wide angle image containing the Hon, thereby tracking the lion. 

23 Thus, the present invention may employ tracking techniques, known in the prior art, to the 

24 obtained digital image. 

25 In the monitoring and tracking embodiment distortion may be removed from the wide 

26 angle image prior to performing the processing to determine whether an event is occurring. The 

27 type of event being monitored and nature of the object being tracked controls whether 
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1 monitoring md/ov tracking may be performed on the distorted or undistorted image. One of 

2 ordinary skill in the art will choose the system best suited for the particular monitored event or 

3 tracked object. 

4 Figure 42 shows a flow diagram of a monitoring and tracking system. The software 

5 necessary to perform the monitoring/tracking functions may be located at the web site or at the 

6 user' s terminal, such as the home system 258 and viewer 266. The image/video signal to be 

7 processed for monitoring and/or tracking may be a live video feed or be played back from 

8 stored video. Thus, a wildlife scientist may leave multiple video cameras running overnight (or 

9 a single video camera with a wide-angle lens) and when the video tape is played back, the 

10 segments/cameras containing activity are displayed. 

1 1 Referring to Figure 42, an "input frame of reference" routine 2700 is executed. This 

1 2 routine is optional, and is used to estabUsh a frame of reference direction, such as north. The 

1 3 frame of reference may determine the first segment of a wide-angle image to view, or the first 

14 camera to view. Next, a "reset segment counter" routine 2710 is executed. This sets the 

15 segment or camera to be first displayed. 

1 6 Each segment or camera is viewed only for a limited time, prior to viewing the next 

1 7 segment or camera. Thus, a "reset timer" routine 27 1 5 is executed to reset the interval when 

18 segments or cameras are switched. 

1 9 Next, the "obtain image" routine 2720 is executed. This routine obtains the wide angle 

20 image (live or prerecorded), or images from all the cameras (in the multiple camera perceived 

2 1 control embodiment of Figures 27 - 28d). The obtained image from a wide-angle lens may be 

22 processed to remove the distortion or not, depending on what is being monitored. 

23 The obtained image is processed to determine active areas (cameras or segments) , 

24 Active areas are areas where the processor determines that activity is taking place, either by 

25 changes in the pixels at those locations, by using other known image/video processing 

26 techniques, or by using external sensors. The processing is performed as known in the art and 

27 is not described further herein. The processing occurs during the "process for activity" routine 
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1 2730. This routine uses the frame of reference to detenrdne which segment(s), relative to the 

2 normal (i.e., north) is/are active. 

3 If activity is present, the "display active segments' ' routine 2750 displays the active 

4 segments or cameras on a display. Distortion from the relevant segments is removed in the 

5 wide-angle lens embodiment. If more than one segment is active, a split screen display may 

6 show each segment simultaneously. The split screen display may make reference to the frame 

7 of reference that was previously entered during routine 2700. The "reset timer" routine 27 10 

8 is then executed so that the last segment under view is returned when activity is no longer 

9 present. 

1 0 If activity is not present, the "display current segment" routine 2760 is executed. This 

1 1 routine displays the current segment or camera until the timer expires, at which point the next 

1 2 segment or camera is displayed. The display may make reference to the frame of reference 

13 which was previously entered during routine 2700. 

1 4 After displaying the current segment or camera, the ' 'time limit exceeded" routine 2770 

15 is executed. If the time hmit has not been exceeded, a branch to the "obtain image" routine 

1 6 2720 occurs and processing continues until the time limit is exceeded, or until activity occurs. 

17 In an "autopan" embodiment (Figure 28a), the time limit value may be increased by pressing 

1 8 the "-" button in conjunction with the "speed" button (Figure 28a), for a slower autopan, and 

19 the time limit may be decreased by pressing the "+" button in conjunction with the "speed" 

20 button (Figure 28a) for a faster autopan. 

21 If the time limit is exceeded, the segment (or camera) counter is incremented by the 

22 "increment segment counted' routine 2780. If the counter is greater than the maximum number 

23 of cameras or segments, the "counter > max" routine 2790 branches to the "reset segment 

24 counter" routine 27 1 0, to restart the automatic panning. If the counter is not greater than 

25 allowed, a branch occurs to the "reset timef routine 27 1 5 so that the next segment or camera 

26 may be displayed, and processing for activity continues. 
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1 Thus, the flow chart of Figure 42 allows for automatic panning and for automatic 

2 tracking. If the "process for activity" routine 2730, the "activity?" test 2740, and the "display 

3 active segments" routine 2750 were removed, the "autopan" function described previously and 

4 shown with respect to Figure 28a - 28d would be achieved, hi this case, "display current 

5 segment" routine 2760 would follow "obtain image" routine 2740. 

6 Monitoring and automatic panning may be combined. When combined, all active 

7 segments or cameras are automatically panned for a brief timeframe. Thus, if a lion and zebra 

8 are both moving towards the camera from opposite direction, each would be displayed for a 

9 brief timeframe before switching to a display of the other. This is an altemati ve to the split 

10 screen display previously described. 

11 1. Display of Video Data 

12 Li the systems described above, the user may select or be provided data concerning 

1 3 the video currently displayed. For example, superimposed on the video may be the date and 

14 time the video was recorded, a name of the image location, remaining time for the video, or 

1 5 data pertaining to the segment (or camera source) of the video which is currendy being viewed 

1 6 This segment/camera data may be a compass heading (such as north) or angle from a 

17 reference (such as 40 degrees), or coordinate information (such as X/Y, XJYIZ, R/0, and 

1 8 X/R/0) relating to the location of the center of the segment/video currenfly displayed in relation 

19 to the wide angle image or other cameras. A graphical representation of the lens (or layout of 

20 the cameras) may show which segment of the wide angle image (or camera) is being displayed. 

21 In order to display the image segment, a frame of reference may be adopted, especially for a 

22 spherical lens . The frame of reference would be either generated by a processor at the web 

23 site or user' s terminal, or entered by a user or operator. For example, the user may select 

24 which direction is "north" or position the axis of a coordinate system if a coordinate display is 

25 to be used for a particular lens. 

26 Additionally, the image' s magnification and its density/colors may also be shown on the 

27 display, such as "magnification = lOx, picture density = 200x200 pixels, 64 colors." 
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1 The display of image data may be used in all embodiments of the present invention, and 

2 are preferably updated when the displayed image changes. 

3 Figure 43 shows a display 2800 showing a coral reef 2805 where users have virtual 

4 camera control using multiple underwater cameras. On the screen 2807, the date 28 1 0 is 

5 displayed along with the time 2820. The location is shown at 2830 and the remaining time of 

6 the program at 2840. The magnification is shown at 2850 and the density and colors at 2860. 

7 The segment camera field 2870 shows that the user is viewing camera no. 3. This 

8 segment/camera data may be shown graphically, as depicted at 2880. Field 2880 is a top view 

9 of the coral reef 2805 and the layout of the cameras, in this case cameras 1 through 10. The 

1 0 square around camera no. 3 indicates that this camera is the source of the picture on the display 

1 1 2800. The frame of reference (north) is indicated at 2890 for the graphical segment data and 

12 2895 for the video data. 

13 J. Storing Video and hiteractive Presentations. 

14 The images, video, and image data may also be stored at the user' s terminal. The wide 

1 5 angle distorted image may be stored, along with the image data, if present. Storage of the 

1 6 image and image data enables the user to retrieve the image and view a segment at a later date. 

1 7 Optionally, the entire interactive presentation may be stored at the user' s terminal (including 

1 8 associated graphics, text, video, data, or other information), although all the pertinent files and 

19 data would have to be received by the user. 

20 ThedisclosureofPCTPublicationNo. WO96/08105,publishedMarch 14, 1996by 

2 1 Labun, incorporated herein by reference is related to storing images and may be used with the 

22 present invention. 

23 The video or image may be stored in either its distorted or undistorted state. Storing 

24 the video or image in its undistorted state has the advantage in that tall and/or wide pictures may 

25 be stored in their most viewable state, and in that editing may be performed on the images more 

26 easily if they are retrieved with the distortion removed. 
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1 The artisan of ordinary skill will appreciate that other aspects of the patent apphcations, 

2 patents and publications incorporated herein by reference may be applied to the present 

3 invention. As such, the patent applications, patents and publications are incorporated herein 

4 in their entirety. The terms and descriptions used herein are set forth by way of illustration only 

5 and are not meant as limitations. Those skilled in the art will recognize that numerous variations 

6 are possible within the spirit and scope of the invention as defined in the following claims. 
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1 CLAMS: 

2 1 . An apparatus for communicating audio and video signals to an electronic book viewer, the 

3 apparatus comprising: 

4 a web site, connected to a plurality of electronic book viewers, comprising: 

5 a receiver that receives digitally compressed audio and video; 

6 an audio- video server that provides a plurality of digital video signals; 

7 a switch that switches and combines the plurality of digital video signals; 

8 an administrative unit, connected to the switch that directs which signals are 

9 switched and combined; and 

10 a transmitter, connected to the switch that communicates the digital audio and 

1 1 video signals to the electronic book viewer as video streams. 

12 2. The apparatus of claim 1, wherein the web site further comprises: 

1 3 an audio and video storage device, connected to the receiver that stores at 

14 least some received audio and video; 

15 a data storage device, connected to the receiver that stores textual and 

16 graphical data; 

17 a database server, connected to the administrative unit, the switch, the audio 

18 and video storage device and the data storage device; 

1 9 wherein the administrative means directs the database server to retrieve and supply to 

20 the switch the audio and video information from the audio and video storage device and the 

21 textual and graphical data from the data storage device. 

22 3 . An apparatus providing the perception of remote camera control to a user, the apparatus 

23 comprising: 

24 a plurality of remote video cameras arranged to film a remote site, each remote video 

25 camera providing a video signal of a different perspective of the remote site; 
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1 a compressor, connected to the video cameras, that compresses the video signals; 

2 a data communication network, connected to a web site and to the compressor, that 

3 carries the compressed video signals from the remote site to the web site; 

4 the web site connected to the communication network and to electronic book viewers, 

5 comprising: 

6 a receiver, having the compressed video signals as its input; 

7 an administrative unit that determines which video signals to pass to a user 

8 terminal in response to a user command; 

9 a switch, controlled by the administrative unit, for switching received video 

10 signals to conrimunication equipment; 

1 1 communication equipment, in operative communication with the administrative 

1 2 unit, for transmitting the switched video signals to user terminals and for receiving a user 

13 command; 

1 4 wherein an electronic book viewer transmits the user command to the web site and the 

1 5 administrative unit directs the switch to provide video signals to the electronic book viewer in 

1 6 accordance with the user command thereby enabling a user to remotely control the position or 

17 orientation of the video signal being received at the electronic book viewer by entering user 

18 commands. 
19 

20 4. A system for providing a user with perceived camera control via an Internet web site, 

21 comprising: 

22 communications equipment to receive camera control commands from one or 

23 more connected users and to transmit video to the one or more connected users; 

24 electronic book viewers to receive the transmitted video and to transmit 

25 camera control conmiands; 

26 video of different views of a remote site; 



-93- 



Docket 52S1/PTO Filings/Spec wpd 

1 an administrative unit, wherein the administrative unit determines which view 

2 of the remote site to transmit to a connected user in response to a received camera 

3 control command, thereby providing the connected user with the perception of camera 

4 control. 

5 5 . The system of claim 4, wherein the system further comprises a video storage unit, 

6 wherein the video storage unit supplies video of different views of the remote site to the web 

7 site, 

8 6. The system of claim 4, wherein the video of different views of the remote site is video 

9 of different camera angles of the remote site. 

10 7. The system of claim 4, wherein the video of different views of the remote site is a 

1 1 distorted wide angle video of the remote site, and wherein the system further comprises a 

12 means for removing distortion from at least one view of the wide angle video. 

13 8. A system for providing a user with actual camera control, the system comprising: 

14 a web site, comprising: 

1 5 communications equipment to receive camera control commands from one or 

16 more connected users and to transmit video to the one or more connected users; 

17 a video receiver for receiving video from a remote video camera; 

1 8 a transmitter in conmiunication with the remote video camera; 

19 a camera control unit, which outputs fomiatted camera control commands to 

20 the transmitter; and 

21 wherein the camera control commands received from a user are formatted and 

22 transmitted to the remote camera and control the remote camera. 
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1 9. A method of remotely viewing a remote site, the method comprising the steps of: 

2 accessing a communications network; 

3 receiving video depicting one or more views of the remote site via the commvmications 

4 network; 

5 entering conmiands regarding a different view of the remote site; 

6 displaying the different view of the remote site. 

7 10. The method of claim 9, wherein the communications network is the Internet, and further 

8 comprising the steps of: 

9 addressing a web site on the Internet; 

10 selecting a remote site. 

11 11. The method of claim 9, wherein the received video is distorted wide angle video, and 

1 2 wherein the step of displaying comprises the step of removing distortion from a segment of the 

13 distorted wide angle video pertaining to the different view to be displayed. 

14 12. The method of claim 9, wherein the received video is video from one of a plurahty of 

15 remote cameras, and further comprising the steps of: 

1 6 processing the entered command to select one of the remote cameras in accordance 

17 with the commanded different view; and 

18 receiving video of the different view from the selected remote camera. 

19 13. The method of claim 12, wherein the displaying step further includes the step of 

20 indicating the location of the selected remote camera and a frame of reference at the remote 

21 site. 
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1 14. The method of claim 13, wherein the step of indicating further comprises the step of 

2 graphically displaying a layout of cameras at the remote site with respect to the frame of 

3 reference. 

4 15. The method of claim 9, wherein the displaying step further includes the step of 

5 indicating the location of a frame of reference at the remote site. 

6 16. The method of claim 9, wherein the displaying step further includes the step of 

7 indicating data concerning the video, the data selected from the group consisting of: remote site 

8 location, remote site time, 

9 17. The method of claim 9, wherein the displaying step further includes the step of 

10 indicating data concerning the video, the data selected from the group consisting of: 

1 1 magnification, pixel density of the video, number of colors in the video. 

12 18. The method of claim 9 , wherein the entered conunand is a command to monitor the 

13 remote site, the method further comprising the steps of: 

14 processing the video for activity at the remote site; 

15 and wherein the step of displaying includes the step of: 

16 selecting views of the remote site displaying activity if activity is present. 

17 19. The method of claim 18, wherein the step of displaying further includes the step of: 

18 automatically panning the remote site if activity is not present. 

19 20. The method of claim 1 8, wherein the received video is wide angle distorted video, and 

20 the step of processing includes the step of removing distortion from at least a portion of the 

21 received video to detect whether activity is present. 



-96- 



Docket 5281 /pro Rlings/Spec wpd 

1 2 L The method of claim 20, wherein the step of selecting includes the step of choosing 

2 segments of the wide angle video for viewing, and the step of displaying further includes the 

3 step of removing distortion from the chosen segments. 

4 22. The method of claim 18, wherein the received video is video from a plurality of 

5 cameras, and the step of selecting includes the step of choosing one or more cameras for 

6 viewing if activity is present. 

7 23 . The method of claim 1 8, wherein the entered command is a conmiand to automatically 

8 pan the remote site, and wherein the step of displaying further includes the step of incrementally 

9 viewing, for a fixed time, a plurality of different views of the remote site. 

10 24. The method of claim 23, further comprising the step of: 

1 1 selecting whether to increase or decrease the fixed time. 

12 25. The method of clam 9, further comprising the steps of: 

13 receiving data and graphics concerning the remote site; 

1 4 and where the step of displaying further comprises the step of showing the data and 

15 graphics. 

1 6 26. The method of claim 25, further comprising the step of saving the video, graphics, and 

17 data in a storage media. 

18 27, The method of claim 9, further comprising the step of saving the video, graphics, and 

19 data in a storage media. 

20 28. An electronic book, comprising: 
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1 a link to an Internet web site, the web site providing a plurality of streaming video, 

2 audio and text data when connected to the electronic book; and 

3 a control function, wherein the control function allows selection of one or more of the 

4 plurahty of streaming video, audio and text data, and wherein the selected data are displayed 

5 with display of the electronic book, 

6 29. The electronic book of claim 28, wherein the electronic book is adapted to be 

7 displayed on an electronic book viewer. 

8 30. The electronic book of claim 28, wherein the electronic book is adapted to be 

9 displayed on a television. 

10 31. The electronic book of claim 28, wherein the electronic book is adapted to be 

1 1 displayed on a personal computer. 

12 32. The electronic book of claim 28, wherein the electronic book is adapted to be 

13 displayed on a palm-sized viewer. 

14 33. The electronic book of claim 28, wherein the control function, comprises: 

15 a camera selection control that allows a user to select a camera angle from which is 

16 provided a video signal; and 

1 7 a multiple screen function that provides for display of video signals from more than one 

18 camera angle. 

19 34. The electronic book of claim 33, wherein the video signals are tiled for display. 
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1 35. The electronic book of claim 33, wherein the a first video signal is provided in a 

2 picture-in-picture format with a second video signal. 

3 36. The electronic book of claim 28, wherein the electronic book is stored on a device 

4 having a memory, and wherein one or more of the streaming video, audio and text data are 

5 stored in the memory. 

6 37 . The electronic book of claim 28, wherein the one or more of the plurality of streaming 

7 video, audio and text data are provided live with display of the link in the electronic book. 

8 38. The electronic book of claim 28, wherein the connection to the Internet web site is 

9 completed using a wired conmiunication system. 

10 39. The electronic book of claim 28, wherein the connection to the Litemet web site is 

1 1 completed using a wireless communication system. 

1 2 40. The electronic book of claim 28, wherein the connection to the Internet web site uses 

13 an electronic link. 

14 41. A computer-readable medium for controlling a connection between an electronic book 

15 viewer and a world watch live site, comprising: 

16 an interface module that receives user inputs, the inputs comprising commands to select 

17 a viewing segment; 

18 a memoiy module coupled to the interface module that directs storage of the received 

19 inputs; 
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1 a transmission module coupled to the memory module that transmits the stored inputs 

2 to a remote location; and 

3 a receiving module that receives video and audio data from the selected viewing 

4 segment. 

5 42 . The computer-readable medium of claim 4 1 , wherein the world watch U ve site is an 

6 Internet web site. 

7 43. The computer-readable medium of claim 41, wherein the commands comprise: 

8 a camera pan command; 

9 a camera select conmiand; 

10 a camera off command; 

11 a zoom command; 

12 a camera on command; and 

13 a camera tilt command. 

14 44. The computer-readable medium of claim 44, further comprising a display module 

15 coupled to the receiving module that displays the selected viewing segment. 

16 45 . The computer-readable medium of claim 44, wherein the receiving module receives 

17 multiple selected viewing segments and the display module displays the received multiple 

18 viewing segments simultaneously. 

19 46. The computer-readable medium of claim 44, wherein the viewing segments comprise 

20 different views of the world watch live site. 
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1 47 . The computer-readable medium of claim 4 1 , wherein one or more of the viewing 

2 segments comprises a wide angle segment, the computer-readable medium further comprising 

3 a video correction module coupled to the display module that removes distortion from the wide 

4 angle segment before display. 

5 48 . The computer-readable medium of claim 4 1 , wherein the video and audio data are 

6 saved in a memory. 

7 49. An electronic book, comprising: 

8 a data file; 

9 a graphics file; 

10 interactive files; 

1 1 electronic links included in the data file and the graphics file, wherein the electronic links 

12 couple the electronic book to the interactive files, the interactive files, including: 

13 an input file to receive commands and data, and 

1 4 an output file to provide prompts and instructions and to provide the input file; 

15 and 

1 6 a control program that operates to couple the electronic book to an interactive file upon 

17 activation of an electronic link. 

18 50. The electronic book of claim 49, wherein the interactive file is displayed on an 

1 9 electronic book viewer. 

20 51. The electronic book of claim 49, wherein the interactive file is a world watch live site. 

2 1 52. The electronic book of claim 49, further comprising an interactive file menu, the 

22 interactive file menu including a link to the interactive file, wherein the menu includes a Ust of 
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1 avaHable interactive files, and wherein activation of the Unk connects the menu to the interactive 

2 file, the interactive file being displayed on a screen of an electronic book viewer. 
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1 ABSTRACT 

2 An electronic book selection and delivery system distributes electronic text and 

3 graphics to subscribers. The system contains an operations center, a video distribution system 

4 or a variety of alternative distribution systems, a home subsystem, and a billing and collection 

5 system. The operations center and/or distribution points perform the functions of manipulation 

6 of text data, security and coding of text, cataloging of books, message center, and uplink 

7 functions. The home subsystem connects to a video distribution system or variety of altemative 

8 distribution systems, generates menus and stores text, and transacts through communicating 

9 mechanisms . A portable book-shaped viewer is used for viewing the text. The viewer can 

1 0 receive digital video data from a variety of sources, including the Intemet by wired or wireless 

1 1 connections. The viewer may use the Intemet connection to control and receive multiple line 

1 2 video feeds such as from a world watch live system. Users may be able to view a plurality of 

1 3 remote locations in real time and remotely control a video picture of a distant location. The 

1 4 remote control may be either actual control of a remote video camera or perceived remote 

15 control by the manipulation of audiovisual data streams. Text, graphics, and other video 

1 6 information supplement one or more video pictures to provide an educational and entertaining 

17 system. Information is accessible to users who are viewing multiple video pictures. The 

1 8 information relates and describes what is being viewed. Users who have different types of 

19 equipment, with different data rates, are able to access and use the system of the present 

20 invention. Users may interactively communicate with a video lecturer by asking questions and 

2 1 receiving answers . 
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I acknowledge the duty to disclose to the United States Patent and Trademark Office all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
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Section 365(c) of any PCT International application designating the United States, listed below and, 
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United States or PCT International application in the manner provided by the first paragraph of 35 
U.S.C. Section 112, I acknowledge the duty to disclose to the United States Patent and Trademark 
Office all information known to me to be material to patentability as defined in Title 37, C. F. R., 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT International filing date of this application: 
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Pending 



(Application Serial No.) 


(Filing Date) 


(Status) 
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Patented 
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(Filing Date) 
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(Filing Date) 


(Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 
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I hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 
application(s) listed below: 



(Application Serial No.) 


(Filing Date) 




(Application Serial No.) 


(Filing Date) 




(Application Serial No.) 


(Filing Date) 




1 hereby claim the benefit under 35 U. S. C. Section 120 of any United States application(s), or 


Section 365(c) of any PCT International application designating the United States, listed below and, 


insofar as the subject matter of each of the claims of this application is not disclosed in the prior 


United States or PCT International application in the manner provided by the first paragraph of 35 


U.S.C. Section 112, 1 acknowledge the duty to disclose to the United States Patent and Trademark 


Office all information known to 


me to be material to patentability as defined in Title 37, C. F. R., 


Section 1 .56 which became available between the filing date of the prior application and the national 


or PCT International filing date of this application: 




08/906,569 


August 5, 1997 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 


09/191,520 


November 13, 1998 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 


08/923,091 


September 4, 1997 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 



1 hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 
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I hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 



appiication(s) listed below: 


(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Filing Date) 



I hereby claim the benefit under 35 U. S. C. Section 120 of any United States application(s), or 
Section 365(c) of any PCT International application designating the United States, listed below and, 
insofar as the subject nnatter of each of the claims of this application is not disclosed in the prior 
United States or PCT International application in the manner provided by the first paragraph of 35 
;J U.S.C. Section 112, I acknowledge the duty to disclose to the United States Patent and Trademark 
it- Office all information known to me to be material to patentability as defined in Title 37, C. F. R., 
m Section 1.56 which became available between the filing date of the prior application and the national 
Q or PCT International filing date of this application: 



09/237,827 


January 27, 1999 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 


09/237,828 


January 27, 1999 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 


09/391,461 


Septembers, 1999 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 
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I hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 



application{s) listed below: 


(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Filing Date) 



I hereby clainn the benefit under 35 U. S. C. Section 120 of any United States application(s), or 
Section 365(c) of any PCT International application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States or PCT International application In the manner provided by the first paragraph of 35 
^3 U.S.C. Section 112, I acknowledge the duty to disclose to the United States Patent and Trademark 
:y Office all information known to me to be material to patentability as defined in Title 37, C, F. R., 
\il Section 1.56 which became available between the filing date of the prior application and the national 
or PCT International filing date of this application: 



09/289,956 


April 13, 1999 


Pending 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 



(Application Serial No.) (Filing Date) (Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 
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POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorney(s) and/or 

agent(s) to prosecute this application and transact all business in the Patent and Trademark Office 

connected therewith, (list name and registration number) 

Note, Aldo, Reg. No, 35,628 

Harrop, John K,, Reg. No. 41,817 

Ryan, John W., Reg. No. 33,771 

Clark, W. Robinson H., Reg. No. 41,530 

Shah, Ami P., Reg. No. 41,143 

Wooden, Sean S., Reg. No. 43,997 

Vietzke, Lance L. Reg. No. 36,708 

Phillips, Matthew C, Reg. No. 43,403 

McWhinney, Christopher, Reg. No. 42,875 



Send Correspondence to: J*'*'" ^- Harrop, Reg. No. 41,817 

DORSEY & WHITNEY LLP 
% 1001 Pennsylvania Avenue, N.W., Suite 300 South 

Washington, D.C, 20004 

111 Direct Telephone Calls to: (name and telephone number) 

□ John K, Harrop, Reg. No. 41,817 (202) 824-8821 





Full name of sole or first inventor 
HENDRICKS, John S. 




Sole or first inventor's signature 


Date 




Residence 

Montgomery County, MD 10854 




Citizenship 
US 




Post Office Address 

8723 Persimmon Tree Road, Potomac, MD 20854 







Full name of second inventor, if any 




McCOSKEY, John S. 




Second inventor's signature 


Date 


Residence 




Douglas County, CO 80104 




Citizenship 




US 




Post Office Address 




4692 N. Lariat Drive, Castle Rock, CO 80104 
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Full name of third inventor, if any 
ASMUSSEN, Michael L. 






Third inventor's signature 


Date 








Residence 

Fairfax County, VA 20171 






Citizenship 
US 






Post Office Address 

2627 Meadow Hall Drive, Herndon, VA 20171 


















Full name of fourth inventor, if any 






Fourth inventor's signature 


Date 








Residence 






Citizenship 






Post Office Address 




f 
















Full name of fifth inventor, if any 






Fifth Inventor's signature 


Date 








Residence 






Citizenship 






Post Office Address 


















Full name of sixth inventor, if any 






Sixth inventor's signature 


Date 








Residence 






Citizenship 






Post Office Address 

















Form PTO-SB-01 (6-95) (Modified) Patent and Trademark Off ice-U.S. DEPARTMENT OF COMMERCE 



